Journal of Advances in Modeling Earth Systems
  • Open Access

Development and verification of a new wind speed forecasting system using an ensemble Kalman filter data assimilation technique in a fully coupled hydrologic and atmospheric model

Authors

  • John L. Williams III,

    Corresponding author
    1. Hydrologic Science and Engineering Program, Department of Geology and Geological Engineering, Colorado School of Mines, Golden, Colorado, USA
    2. Meteorological Institute, University of Bonn, Bonn, Germany
    Current affiliation:
    1. Research Applications Laboratory, National Center for Atmospheric Research, Boulder, Colorado, USA
    • Corresponding author: J. L. Williams, Meteorological Institute, University of Bonn, Meckenheimer Allee 176, DE-53115 Bonn, Germany. (jwilliam@uni-bonn.de)

    Search for more papers by this author
  • Reed M. Maxwell,

    1. Hydrologic Science and Engineering Program, Department of Geology and Geological Engineering, Colorado School of Mines, Golden, Colorado, USA
    2. Integrated Groundwater Modeling Center, Colorado School of Mines, Golden, Colorado, USA
    Search for more papers by this author
  • Luca Delle Monache

    1. Research Applications Laboratory, National Center for Atmospheric Research, Boulder, Colorado, USA
    Search for more papers by this author

Abstract

[1] Wind power is rapidly gaining prominence as a major source of renewable energy. Harnessing this promising energy source is challenging because of the chaotic nature of wind and its inherently intermittent nature. Accurate forecasting tools are critical to support the integration of wind energy into power grids and to maximize its impact on renewable energy portfolios. We have adapted the Data Assimilation Research Testbed (DART), a community software facility which includes the ensemble Kalman filter (EnKF) algorithm, to expand our capability to use observational data to improve forecasts produced with a fully coupled hydrologic and atmospheric modeling system, the ParFlow (PF) hydrologic model and the Weather Research and Forecasting (WRF) mesoscale atmospheric model, coupled via mass and energy fluxes across the land surface, and resulting in the PF.WRF model. Numerous studies have shown that soil moisture distribution and land surface vegetative processes profoundly influence atmospheric boundary layer development and weather processes on local and regional scales. We have used the PF.WRF model to explore the connections between the land surface and the atmosphere in terms of land surface energy flux partitioning and coupled variable fields including hydraulic conductivity, soil moisture, and wind speed and demonstrated that reductions in uncertainty in these coupled fields realized through assimilation of soil moisture observations propagate through the hydrologic and atmospheric system. The sensitivities found in this study will enable further studies to optimize observation strategies to maximize the utility of the PF.WRF-DART forecasting system.

1. Introduction

[2] Without effective options for energy storage, providers of wind energy must rely heavily on accurate short-term forecasts of wind speed to effectively maximize the proportion of energy captured from wind in the load of existing power grids [Porter and Rogers, 2010]. Both numerical and statistical models have been applied to the problem of weather forecasting, each with strengths relevant to wind forecasts over different time scales. Statistical models provide very accurate and computationally inexpensive probabilistic wind speed forecasts over short time scales (on the order of a few hours) using a combination of observed data and persistence; however, their accuracy tends to degrade quickly with increasing forecast lead time [e.g., Brown et al., 1984; Kretzschmar et al., 2004]. Gneiting et al. [2006] and Hering and Genton [2010] fine-tuned these techniques by incorporating directional components into the predictive function adding a physical component to the forecast. Numerical weather prediction more robustly incorporates a physical representation of the system and while computationally more expensive than statistical models, can provide skillful forecasts over longer periods of time with applications ranging from long-term global climate prediction to short-term, high-resolution localized wind forecasts [Giebel et al., 2003], even over complex terrain [Clark et al., 1997; Grønås and Sandvik, 1999]. Combining statistical and numerical forecast tools into statistically tuned physical representations of the atmospheric system can further improve wind forecasts by applying statistical post processing techniques to numerical ensemble outputs to generate sharpened and more reliable probabilistic forecasts [e.g., Gel et al., 2004; Hamill and Whitaker, 2006; Berrocal et al., 2007; Pinson and Madsen, 2009; Sloughter et al., 2010; Delle Monache et al., 2013]. Ensemble simulations, in general, add a probabilistic component that can be used to evaluate the uncertainty of a forecast and are used operationally by, for example, the European Centre for Medium Range Weather Forecasts [Molteni et al., 1996] and the National Center for Environmental Prediction [Toth and Kalnay, 1997], and in more specialized applications such as flood forecast models [e.g., Bao et al., 2011]. Monte Carlo techniques are common in hydrological applications, described in stochastic hydrolgeology textbooks [e.g., Rubin, 2003], and used in advanced applications such as multimodel hydrologic ensembles to enhance the skill of predictions with Bayesian model averaging schemes [Ajami et al., 2007; Duan et al., 2007].

[3] The primary limitations in using numerical weather prediction to generate forecasts are the computational expense (a high-resolution forecast over a large spatial extent often requires the resources of high-performance computing environments) and model error stemming from simplifications and parameterizations in the numerical model's physical representation of the system it simulates [Hanna and Yang, 2001; Eckel and Mass, 2005]. Improvements in numerical models can be made by replacing assumptions and parameterizations with the mathematical representations of the physical processes they simulate. Of interest in this work is the representation in the modeling system of interactions between the land surface and the atmosphere. Work by Chen and Avissar [1994] clearly demonstrated a strong connection between soil moisture distribution on the land surface and regional scale precipitation and wind patterns. Patton et al. [2005] showed more specifically that heterogeneous surface soil moisture patterns are connected with very different development in the atmospheric boundary layer when compared with homogeneous land surface initializations, and Holt et al. [2006] further demonstrated that physically based land surface schemes that better represent vegetative processes and soil moisture initializations tend to yield more accurate forecasts of regional atmospheric disturbances. To capture the land surface-atmosphere physical connection, Maxwell et al. [2011] developed a fully coupled hydrologic and atmospheric model using the ParFlow (PF) hydrologic model [Ashby and Falgout, 1996; Jones and Woodward, 2001; Kollet and Maxwell, 2006; Maxwell, 2013] and the Weather Research and Forecasting (WRF) model [Skamarock and Klemp, 2008; Skamarock et al., 2008] in order to physically represent hydrologic processes from bedrock to the top of the atmosphere. This fully coupled model, PF.WRF, was used by Williams and Maxwell [2011] to show that reduction in uncertainty in subsurface characterization of hydraulic conductivity propagates into atmospheric variables, specifically wind speed.

[4] Assimilation of observations into forecast models represent another avenue by which improvements can be made to the forecasts produced by numerical simulations. In operational settings, 3-D and 4-D variational analysis (3D-Var and 4D-Var) are commonly used to assimilate observed quantities into the forecast model state, assuming an isotropic and static spatial distribution of error statistics. The adjoint of the forward model can be applied in these least squares techniques allowing for time evolution of the error statistics, in the case of 4D-Var. This Bayesian technique is not used in 3D-Var, which provides a computational savings [Lorenc et al., 2000; Lorenc, 2003]. The Kalman filter and its extension the extended Kalman filter (detailed in section 2) eliminate the need for linear and adjoint models to be applied to the error covariance calculations and also eliminate the assumption of isotropy and stationarity [Lorenc, 2003; Hamill, 2006]. The application of Monte Carlo techniques makes the Kalman filtering technique more tractable with the ensemble Kalman filter (EnKF) [Evensen, 1994; Anderson and Anderson, 1999; Anderson, 2001; Evensen, 2003]. Adding a linear operator to the ensemble Kalman filter to reduce the unrestricted propagation of error due to artificial added noise through the system yields better results, allowing for smaller ensembles [Anderson, 2001], though as we will show, larger ensembles are preferable and provide better results. Evensen's [1992] experiments with the extended Kalman filter and the ensemble Kalman filter [Evensen, 1994] were focused on oceanographic applications, but it is also used for meteorological applications [Lakshmivarahan and Stensrud, 2009]. Houtekamer and Mitchell [1998] experimented with the error covariance characteristics of the ensemble Kalman filter in an atmospheric ensemble assimilating satellite and radiosonde observations using a dual filter technique where two ensembles are used to generate error statistics and the error statistics of one ensemble are applied to the calculations of the other in an effort to reduce the effects of statistical “inbreeding.” The ensemble Kalman filter has also been applied to hydrologic applications ranging from back calculation of porosity and hydraulic conductivity from head and concentration measurements [Li et al., 2012] to assessment of variably saturated groundwater flow regimes to track contaminant transport [Hendricks-Franssen et al., 2011; Kollat et al., 2011]. Reichle et al. [2002] used the ensemble Kalman filter to update soil moisture fields in a land surface model forced with offline atmospheric data.

[5] To date, a robust data assimilation scheme has not been applied to a coupled hydrologic and atmospheric modeling system. Wang et al. [2012] used 3D-Var to integrate observations in a coupled flood forecasting system employing atmospheric and hydrologic models coupled in a one-way offline scheme (i.e., atmospheric outputs force the hydrologic model). The PF.WRF model is coupled online such that feedbacks between the atmosphere and the surface and subsurface are simulated. This is described more in section 2. Using the EnKF functionality provided by the Data Assimilation Research Testbed (DART) [Anderson et al., 2009], an open source software package that includes a selection of several algorithms to assimilate observational information (and contains interfaces for several models including WRF), we extend the existing WRF-DART framework to include the dynamic hydrology of ParFlow, improving the physical representation of the simulated system and incorporating observed data to improve the predictive skill in a numerical weather forecast. We use this fully coupled data assimilation framework to study interactions between subsurface processes and the atmospheric processes coupled with them, which can be traced by analyzing the propagation of uncertainty reduction from soil moisture fields through surface energy fluxes to downwind measurements of wind speed.

[6] We selected the EnKF as the data assimilation technique for the fully coupled modeling system because of its flexibility using a Bayesian approach to calculating a flow-dependent forecast error covariance matrix using prior information from the model state vector derived from ensemble error statistics. By using an ensemble technique as opposed to a single realization forecast, these error statistics are easier to calculate as needed at each observation update cycle, resulting in computational efficiency. We have extended the existing WRF-DART interface so that it can work with the fully coupled PF.WRF model and assimilate hydrologic variables not included in the standard WRF-DART interface. This paper presents the methodology for constructing the fully coupled hydrologic and atmospheric modeling system with advanced data assimilation algorithms and describes the process for verifying it by assimilating soil moisture observations in an idealized test case to evaluate the responses of the model to the observations assimilation.

2. Methods

[7] The effort described in this paper represents the development of a method for applying advanced data assimilation techniques, specifically the EnKF, to a fully coupled hydrologic and atmospheric model. We present the verification stage of the fully coupled PF.WRF-DART forecasting system, developed specifically for short-term forecasts of wind speed for wind energy applications, though its utility is not restricted to this application. This system could be applied to other problems such as solar energy, aviation, transport, and dispersion for national security applications, or regional and local weather prediction, in addition to its development testbed of wind energy forecasting.

2.1. Coupled Modeling System

[8] The ParFlow parallel hydrologic model solves flow in the subsurface using a 3-D form of Richards' [1931] equation and includes surface runoff routing using the kinematic wave approximation for the shallow water equations as part of an overland flow boundary condition [Kollet and Maxwell, 2006]. ParFlow can optionally run with the Common Land Model to simulate surface moisture and energy fluxes and plant transpiration processes, and it has also been coupled with the WRF atmospheric model (using the Noah land surface model) allowing a single coupled model to simulate aspects of the hydrologic cycle in the subsurface, on the land surface, and in the atmosphere.

[9] The PF.WRF-DART system uses the Advanced Research WRF (ARW) dynamical core of the WRF model. WRF-ARW is a numerical weather prediction model that explicitly solves the fully compressible nonhydrostatic Navier-Stokes equations in three dimensions tracking velocity, thermal energy, and optionally, moisture and turbulent kinetic energy through second- and third-order Runge-Kutta time integration schemes. The WRF system contains several packages providing options for parameterizations of model physics including cloud microphysics schemes, cumulus schemes, longwave and shortwave radiation formulations, boundary layer parameterizations, and land surface models. With the exception of the land surface formulation, these physics options are interchangeable in PF.WRF and in the PF.WRF-DART system.

[10] ParFlow and WRF are coupled through the Noah land surface model, one of several options for land surface formulations available in the WRF package. Figure 1a shows a schematic of the PF.WRF work cycle that occurs at each time step. Noah calculates moisture and energy fluxes across the land surface as a vegetation stress function responding to atmospheric forcing. These fluxes are limited by potential evaporation, calculated using a Penman-based energy balance approach with stability-dependent aerodynamic resistance [Chen and Dudhia, 2001] over a four-layer soil column and a land cover parameterization based on lookup tables for specified cell-by-cell soil and land cover types. Noah uses a simplified model for hydrologic processes in which moisture movement in a vertically homogeneous soil column is assumed to be exclusively vertical escaping the bottom of the domain by gravity only with no water table dynamics. In this model, precipitation in excess of infiltration is treated as runoff, which is not routed but is simply removed from the system. By coupling this system with ParFlow, this simplified hydrologic formulation is replaced with the fully dynamic 3-D Richards' equation formulation, which is solved in terms of pressure head, h

display math(1)

where

display math(2)
Figure 1.

(a) The PF.WRF model work cycle at each time step begins with WRF and Noah advancing the atmospheric and land surface portions of the model and sending precipitation − evaporation as a flux term to the ParFlow model (called as a subroutine), which then calculates an updated pressure field. The pressure field translates to soil moisture using Van Genuchten's [1980] equation relating pressure head to saturation, which is returned to the Noah LSM for the next advance of WRF and Noah. (b) The PF.WRF model work cycle as shown in Figure 1a, but with a data assimilation update cycle added using DART and an interface to reinitialize ParFlow in addition to WRF at each update cycle.

[11] In equations (1) and (2), Ss is the specific storage, Sw is the relative saturation (calculated using the Van Genuchten [1980] equations relating saturation to pressure head), ϕ is the porosity, qr is a source/sink term (which includes precipitation and evapotranspiration), Ks(x) is the saturated hydraulic conductivity tensor, kr is the relative permeability, v is the subsurface flow velocity, and q is the specific volumetric flux. Pressure head is converted to saturation (internally in ParFlow) using the Van Genuchten empirical relationship, and these saturation values represent the moisture exchange with the land surface. The four-level soil column is no longer restricted to default soil level thickness and can be modeled heterogeneously in all three spatial directions. Furthermore, the addition and removal of water from the system are controlled by the boundary conditions of the hydrologic model allowing for internal vadose and saturated zone dynamics and surface runoff routing [Maxwell et al., 2011].

[12] PF.WRF has been updated to WRF version 3.3, which includes a parameterization for wind farm drag in the lower atmosphere [Fitch et al., 2012]. This eliminates the need to build backward compatibility into DART, which is already compatible with WRF versions 3.1 and later, but not version 3.0. The new version of PF.WRF was subjected to water balance test identical to that from Maxwell et al. [2011], where a positive moisture tendency of 1.0 was added to the water vapor mixing ratio in the atmosphere in the middle of a 15 × 15 km model domain for the first 24 h of a 48 h simulation inducing rainfall and runoff. The second half of the simulation is a drying phase. Water balance, calculated as the sum of surface and subsurface storage minus the sum of runoff and moisture flux across the surface (positive moisture flux represents evaporation and transpiration, negative precipitation), is used to assess the model's ability to keep track of water in the hydrologic part of the system with the atmosphere as a source/sink. Using this metric, the difference between change in storage and the sum of runoff and surface flux should be minimized (and ideally zero). Through the 48 h test simulation, the water balance error as a fraction of total storage reached a maximum error of 2 × 10−15, for a cumulative error of approximately −2.8 mL, over a 225 km2 domain.

2.2. Ensemble Kalman Filter Data Assimilation

[13] A data assimilation capability has been added to the PF.WRF modeling system by interfacing it with DART, which provides several state-of-the-science ensemble-based algorithms for merging numerical predictions with observational data, to provide an improved estimate of the state of the coupled groundwater-land-surface-atmosphere physical system. Those pertinent to this study are the EnKF [Evensen, 1994] and the ensemble adjustment Kalman filter (EAKF), a modified version of the original EnKF [Anderson, 2001]. The DART package, which provides both, includes an interface for WRF; however, this interface is restricted to atmospheric variables. In order to use DART with the PF.WRF system, additional interface components needed to be constructed, and some of the WRF-DART components have been modified so that the system could assimilate soil moisture observations and generate updated fields to reinitialize ParFlow. A flow chart showing the updated work cycle of PF.WRF with DART is shown in Figure 1b.

[14] The EnKF is an extension of the extended Kalman filter—itself an extension of the common Kalman filter described by Gelb [1974]—allowing it to be applied to nonlinear systems [Evensen, 1992]. Each formulation of the Kalman filter operates in two stages: an update stage, where new observations adjust the model state and error statistics, and a propagation step, where the model state and the error statistics are advanced to the next update cycle and model error accumulates. The model state vector is updated at time t, by

display math(3)

where xa is the updated state vector, xp is the prior state vector, y is the set of observations, and H is the forward operator that projects model estimates in the observation space. The update to the state vector is weighted by the Kalman gain matrix,

display math(4)

where Pp is the forecast (background) error covariance matrix and R is the observation error covariance matrix. The updated background error covariance matrix is calculated by

display math(5)

[15] Equations (3)-(5) represent the update stage of the Kalman filter. Equations (6) and (7) propagate the model state and the error statistics to the next update cycle. Propagating the model state is achieved simply by advancing the model starting with the updated state xa,

display math(6)

where Ψ represents, in this case, the PF.WRF model. The background error covariance matrix is advanced as function of an estimate of the model Jacobian, or the transition matrix M,

display math(7)

where Q is the model error accumulated between update cycles [Evensen, 1992, 1994; Houtekamer and Mitchell, 1998; Hamill, 2006].

[16] With a single model, the error statistics must be propagated with the model and calculated internally from the model. A new background error covariance matrix must be calculated each time observations are to be assimilated. The error statistics are estimated at the beginning of the model run, and each time they are updated, moments of third and higher order are discarded from the approximation. The resulting unbounded variance growth in combination with the extra computing power requirement for propagating the error statistics makes the extended Kalman filter problematic [Evensen, 1994]. Using an ensemble forecast system provides a simpler and less computationally demanding method for calculating the background error covariance matrix, eliminating the need to propagate the error statistics with the model state. Instead, the covariances can be calculated directly from the ensemble, from which uncertainty can be easily estimated. The EnKF eliminates the need to solve Equation (7), instead calculating the background error covariance matrix from the ensemble statistics [Evensen, 1994].

[17] The EnKF essentially works as an ensemble of extended Kalman filters, with each ensemble member using its own mean and covariance to describe the error statistics in the calculation of the background error covariance matrix. Because observations and model state are perturbed with artificial noise, artifacts of the noise can be mistaken as correlations in unrelated variables when the error covariance matrices are calculated. The EAKF adds a linear operator to the traditional EnKF calculation of the background error covariance matrix that uses the global ensemble mean and global ensemble covariance in place of those for the individual ensemble members to reduce propagation of this type of error, especially in small ensembles [Anderson, 2001]. This modified EnKF is implemented in DART and used in this verification study because preliminary tests with it demonstrated significantly reduced variance compared with the traditional EnKF.

2.3. Test Case

[18] In order to verify the PF.WRF-DART system, we ran a series of test cases based on the simulations in Williams and Maxwell [2011] and using a re-creation of those simulations as a control for comparison with the outputs of the PF.WRF-DART test runs. Similar to the previous work, simulated observations were collected from a synthetic truth run that was excluded from the model ensemble, but run with the same model setup.

[19] Each simulation, including the control and synthetic truth cases, was run using the PF.WRF version developed as part of this study (WRF version 3.3, rather than version 3.0 as used in Williams and Maxwell [2011]). In the horizontal, the model domain is a planar square 15 km on a side and oriented north-south with a 1000 m grid resolution sloping 0.001 downward toward the west. The subsurface is 5 m thick discretized in 0.5 m layers with the initial water table placed at the bottom of the domain. The system is initialized with the application of uniform rainfall over the entire surface of the domain at a rate of 2 cm/h for a period of 3 h before the 24 h dry-out and observation period starts. Hydraulic conductivity in the subsurface in each ensemble member and the control case is statistically represented as a stochastic field generated using the turning bands random field generator algorithm [Tompson et al., 1989] with identical global statistics (listed in Table 1) but different random number seeds. This produces a unique hydraulic conductivity field for each ensemble member, but the statistics are the same. The entire domain uses constant soil wetting and drying parameters for the Van Genuchten function for unsaturated moisture content [Van Genuchten, 1980], also listed in Table 1. Note that this ensemble approach is somewhat different than the rich literature of atmospheric studies where the model initial conditions are varied to generate the ensemble members [e.g., Lewis, 2005; Leutbecher and Palmer, 2008]. In this approach, which is consistent with a broad range of studies that appear in the hydrogeology and contaminant transport literature [e.g., Criminisi et al., 1997; Nowak et al., 2010], we vary the statistical representation of the hydraulic conductivity fields that characterize the subsurface of each ensemble member. This allows each ensemble member to develop distinct soil moisture initializations through the wetting period preceding the dry-out period analyzed in this work. A standard atmospheric technique of adding low-level artificial noise to the initial conditions in the ensemble has been applied as well, which is consistent with using the EnKF [Anderson, 2001].

Table 1. Settings and Parameterization Selections for WRF and ParFlow
 Value
Domain geometry
Domain size in x (m)15,000.0
Domain size in y (m)15,000.0
Domain size in z (m)5.0
NX15
NY15
NZ10
DX (m)1000.0
DY (m)1000.0
DZ (m)0.5
Slope in x0.001
Slope in y0
Top of atmosphere (Pa)14,960
Atmospheric vertical discretization25 Eta levels
Random field generator settings
Correlation length X (m)5000.0
Correlation length Y (m)5000.0
Correlation length Z (m)2.0
Geometric mean K (m/h)0.01
Variance ln(K)2.25
Number of lines100
R Zeta5.0
Maximum K (m/h)100.0
Del K0.2
Hydrologic parameters
Specific storage1.00E−005
Porosity0.25
Manning's coefficient1.00E−006
Initial water table depth (m)2.00E+000
van Genuchten parameters
Alpha6.0
N2.0
Residual saturation0.2
WRF model physics
MicrophysicsPurdue Lin et al.
Longwave radiationRRTM
Shortwave radiationMM5 shortwave
Surface layerMonin-Obukhov (modified opt 2)
Land surface modelNoah
CumulusNone
PBLNone
Diffusiondiff_opt = 2, km_opt = 2
isfflx1 (drag and heat from physics)
DampingW-Rayleigh (damp_opt = 3)
zDamp5000
Damping coefficient0.01

[20] The atmosphere uses the same horizontal discretization as the land surface and subsurface; however, it is discretized vertically in terms of 25 transient pressure levels fractionally defined from a nominal 200 m near the surface to 800 m at the top of the atmosphere, which is denoted by a minimum pressure of 14,960 Pa. The atmosphere is initialized with 50% relative humidity throughout, a slightly stable temperature profile where temperature t in kelvin is defined as

display math(8)

where z is the elevation above the land surface, and a hydrostatic pressure profile based on a 300 K temperature. The model setup uses periodic lateral boundary conditions. Additional settings for physics and dynamics settings are listed in Table 1.

[21] The control case includes four complete ensembles, replicating the study in Williams and Maxwell [2011]. The first is a 10-member ensemble with statistically equivalent but spatially distinct subsurface hydraulic conductivity fields for each ensemble member. For each subsequent ensemble, the hydraulic conductivity field is conditioned with an increasing number of observed values sampled from the synthetic truth case. Figure 2 shows a plan view of a typical hydraulic conductivity field with the locations of the sampling locations marked. The three conditioned cases comprise 12, 24, and 40 sampling locations with 60, 120, and 200 samples collected, respectively (five samples in the vertical at each location).

Figure 2.

Plan view of a typical hydraulic conductivity field. Black dots mark sampling locations for both hydraulic conductivity values (of which there are five samples in the vertical) and soil moisture values. Labeled regions EAST and WEST mark the observation areas used for the analysis. Labeled point OB was used for the single point observation tests.

[22] Hydraulic conductivity is represented as a spatially correlated random field. Using a natural log transform of hydraulic conductivity

display math(9)

values in the hydraulic conductivity field are represented by the sum of the expected value and a perturbation, y′:

display math(10)

[23] The geometric mean of hydraulic conductivity represents the expected value,

display math(11)

and the expected value of the perturbation

display math(12)

[24] The square of the perturbation is equal to the field variance,

display math(13)

which in the unconditioned case is applied to the spatial correlation structure, defined in terms of the covariance,

display math(14)

where r is the separation distance between point pairs, and λ is the correlation length. The subscript d represents the Cartesian direction (i, j, and k) [Rubin, 2003].

[25] In the conditioned cases, simple kriging is used to tie the perturbation value, y′, to observed values with a weighted correlation structure. Equation (10) becomes

display math(15)

for n observed data points. The weighting factor, ξ, is determined by calculating the minimum error variance at each point for n observed data values

display math(16)

where inline image is the field variance, and C is the correlation structure defined in equation (14). The weighting factor, therefore, is dependent on the correlation length, and thus, the influence of an observed value diminishes with increasing distance. Conditioning by this technique enforces observed values at their spatial locations [Goovaerts, 1997; Rubin, 2003].

[26] The PF.WRF-DART verification test cases were intended to test the extent to which assimilating soil moisture observations affects the coupled variables latent heat flux and wind speed magnitude in the atmospheric pressure layer nearest the land surface. We also tested the effects of increasing the quantity of observations—testing 1, 12, and 40 observations—and the number of ensemble members repeating each run with 10 and 120 ensemble members. Finally, we tested the effect of conditioning subsurface hydraulic conductivity in concert with assimilating observations of soil moisture at the surface using soil moisture values collected at one sample location and hydraulic conductivity values sampled from 12, 24, and 40 surface locations, each with five samples in the vertical. This test was completed with a 120-member ensemble. The experimental cases analyzed in this study are summarized in Table 2.

Table 2. List of Experiments
ExperimentNumber of Ensemble MembersSoil Moisture Observation PointsK Conditioning Points
  1. a

    Replicated from Williams and Maxwell [2011].

CONTROLa1000
COND60a10060
COND120a100120
COND200a100200
ONE-SMC/101010
ONE-SMC/12012010
12SMC120120
40SMC120400
COND200/ONE-SMC1201200

[27] For each test, the DART EnKF updates the model state with soil moisture observations every 6 h. We examine the ensemble averages and 95% confidence interval envelopes in comparison with the control values for soil moisture in the surface soil layer, latent heat flux at the surface, and cell centered resolved wind speed magnitude in the lowest atmospheric pressure level averaged over the areas labeled “WEST” and “EAST” in Figure 2. We also examine the average ensemble spread and the average ensemble mean root-mean-squared error (RMSE) through the 24 h simulation period for each of these variables. Ideally, these quantities will be equal through time in a perfectly statistically consistent ensemble [Talagrand et al., 1997]. Soil moisture and latent heat flux are evaluated in the WEST location because the effects of data assimilation are expected to be more localized for those variables than for wind speed, which are analyzed in the EAST region.

3. Results and Discussion

[28] To evaluate the PF.WRF-DART system, we regenerated the experiment conducted in Williams and Maxwell [2011], which uses a typical hydrologic ensemble approach, as a basis for comparison with the results of the test cases run in this study, in which the ensemble techniques are more typical of atmospheric simulations. Figure 3 shows the synthetic truth, ensemble average, and 95% confidence (twice the standard deviation) envelope for the CONTROL case averaged over the WEST region, with no conditioning of hydraulic conductivity fields and no assimilated observations. Figure 4 shows the corresponding average ensemble spread and average ensemble mean RMSE for soil moisture (Figure 4a) and latent heat flux (Figure 4b). These provide a baseline to which the results of the data assimilation paradigms may be compared.

Figure 3.

Time series of CONTROL case results averaged over region WEST in Figure 2 showing the ensemble mean, synthetic truth, and 95% confidence interval. Soil moisture in the surface soil layer is shown in Figure 3a, and Figure 3b is latent heat flux.

Figure 4.

As in Figure 3 but showing average ensemble spread and average ensemble mean RMSE.

[29] In the ONE-SMC/10 and ONE-SMC/120 test cases, we observe the effect of using the DART EAKF algorithm to assimilate soil moisture observations sampled from a single point, OB (see Figure 2), on simulated soil moisture and latent heat flux averaged over the WEST region, located in the vicinity of the sampling point. Figure 5 shows time series of each of these variables with their 95% confidence envelopes and the synthetic truth for runs with both 10 and 120 ensemble members. Figure 6 shows the average ensemble spread and average ensemble mean RMSE values through time for the same cases. The larger ensemble provides a reduced RMSE of the mean (likely the random component) in both soil moisture and latent heat flux compared with the 10-member ensemble, and the difference between average ensemble spread and average ensemble mean RMSE is smaller in the larger ensemble as well, indicating an improved statistical consistency. The reduced sampling error associated with the larger ensemble makes it preferable because it provides more statistically reliable results.

Figure 5.

Time series of (a and c) soil moisture and (b and d) latent heat flux produced with PF.WRF-DART averaged over region WEST (Figure 2) showing the ensemble mean, synthetic truth, and 95% confidence interval with a 6 h update cycle and one soil moisture observation sampled from point OB. A 10-member ensemble is shown in Figures 5a and 5b (case ONE-SMC/10), and a 120-member ensemble in Figures 5c and 5d (case ONE-SMC/120).

Figure 6.

As in Figure 5 but showing average ensemble spread and average ensemble mean RMSE.

[30] Figure 7 shows a comparison of the average ensemble spread and average ensemble mean RMSE for soil moisture and latent heat flux when observations of soil moisture are assimilated at 1, 12, and 40 locations (ONE-SMC/120, 12SMC, and 40SMC cases). These metrics are calculated for the WEST region, in the vicinity of the assimilated observation location. The difference between the average ensemble spread and average ensemble mean RMSE is smallest for both soil moisture and latent heat flux when soil moisture observations are assimilated at only one location. When additional locations are added, the performance of the EAKF on these variables diminishes. This is likely the result of holding hydraulic conductivity static while adjusting values for soil moisture. Since soil moisture is dependent, in part, on hydraulic conductivity, and latent heat flux is related to soil moisture in moisture-limited hydrologic regimes such as the one modeled here, adjusting soil moisture while keeping hydraulic conductivity static could create a physical conflict that is not resolved when these related quantities are not allowed to freely covary. Averaged over an area such as the WEST region, this effect becomes more pronounced with additional observation locations as more of these physical inconsistencies are introduced into the model state.

Figure 7.

Comparison of time series of average ensemble spread and average ensemble mean RMSE in region WEST for (a, c, and e) soil moisture and (b, d, and f) latent heat flux with soil moisture observations collected from (a and b) 1, (c and d) 12, and (e and f) 40 surface locations, cases ONE-SMC/120, 12SMC, and 40SMC, respectively.

[31] We examine wind speeds averaged over the EAST region in the lowest pressure level, which extends from the ground surface to a nominal altitude of 200 m. Wind speeds in this region are shown in Figure 8, with soil moisture observations assimilated from 1 (a and b), 12 (c and d), and 40 (e and f) surface locations. Assimilating soil moisture observations from additional locations does not appear to significantly affect wind speeds in the downwind EAST region. The difference between average ensemble spread and average ensemble mean RMSE does not reduce substantially when additional sampling locations are added, nor does there appear to be any appreciable reduction in variance or bias. Compared with the unconditioned case with no assimilation of observations (CONTROL, shown in Figure 9), there are a small bias reduction and a small reduction in the difference between average ensemble spread and average ensemble mean RMSE when soil moisture is assimilated at one location, but the addition of more soil moisture observation locations on the upwind side of the model domain does not appear to substantially impact the wind speeds recorded in the EAST region when compared with the result using only one sampling location.

Figure 8.

Time series of (a, c, and e) wind speed in region EAST (in Figure 3) showing the ensemble average, synthetic truth, and 95% confidence intervals and (b, d, and f) average ensemble spread and average ensemble mean RMSE. Figures 8a and 8b show the results with soil moisture observations from one surface location assimilated (point OB), Figures 8c and 8d assimilate soil moisture observations from 12 surface locations, and Figures 8e and 8f from 40 surface locations, cases ONE-SMC/120, 12SMC, and 40SMC, respectively.

Figure 9.

Wind speeds in region EAST for the unconditioned and unassimilated CONTROL case. Synthetic truth, ensemble mean, and 95% confidence interval are shown in Figure 9a, and average ensemble spread and average ensemble mean RMSE are shown in Figure 9b.

[32] In terms of a data collection strategy, little to no improvement is realized in simulated wind speed in the EAST by providing additional soil moisture information. At the same time, higher density of sample locations for surface soil moisture appears to negatively bias the simulated soil moisture values produced in the model ensemble. When the number of observations assimilated increases, the dimensions of the observation error covariance matrix and the Kalman gain matrix also increase. In order to evaluate these and apply them in the EnKF framework, the size of the ensemble must increase as well. When we must use 120 ensemble members to assimilate a single observation of soil moisture, it follows that the ensemble size required to assimilate observations from multiple locations would be much larger to overcome this negative bias introduced with higher observation density. This is amplified by the restrictions placed on the degrees of freedom in the assimilation by holding hydraulic conductivity fields static. These results indicate that a high density of soil moisture observations is not critical in this test case, and the benefit of increasing the observation density does not appear to justify the increase in ensemble size and the corresponding increase in computational expense.

[33] In the final test case (COND200/ONE-SMC), we repeated the single observation experiment in 120-member ensembles using conditioned hydraulic conductivity fields to evaluate the cumulative effect of subsurface characterization and assimilation of soil moisture observations on wind speed downwind of the sampling location, again assimilating soil moisture observations at point OB and observing wind speed in region EAST. Hydraulic conductivity fields are conditioned with 200 sampled values from 40 surface locations (five samples for each surface location). Figure 10 shows a comparison of wind speeds (ensemble mean, synthetic truth, and 95% confidence interval on the left, and average ensemble spread and average ensemble mean RMSE on the right) averaged over the EAST region for three cases: (a and b) hydraulic conductivity field conditioned by 200 samples (COND200); (c and d) soil moisture observations taken from a single sampling location, but no conditioning of hydraulic conductivity fields (ONE-SMC/120); and (e and f) both a single soil moisture observation location and conditioned hydraulic conductivity fields (COND200/ONE-SMC). Each case shows a small reduction in the difference between average ensemble spread and average ensemble mean RMSE when compared with the unconditioned and unassimilated case shown in Figure 9, but this difference is relatively comparable in all three cases. The variance is lowest for the case where hydraulic conductivity fields are conditioned by 200 samples but soil moisture values are not assimilated. The difference between average ensemble spread and average ensemble mean RMSE is comparable for all three cases. Integrating this difference over time provides a scalar quantity with which to rate these forecast regimes relative to each other, and a lower value indicates a stronger spread-skill relationship. For the unconditioned, unassimilated case, the value of this integrated metric is 14.20. Assimilating soil moisture observations at a single location reduces this metric to 13.13. Conditioning the subsurface with 200 hydraulic conductivity samples (without assimilating soil moisture observations) yields a value of 12.88. Using both a 200-point conditioned subsurface and assimilating soil moisture observations at one location yield a value of 13.09. The combination of using conditioned hydraulic conductivity fields in concert with soil moisture observations does not appear to provide a substantial improvement over using the soil moisture observations by themselves. Subsurface conditioning yields the lowest value for this metric, but it is not significantly less than the values for the single observation location or the combined conditioning and EnKF case. It appears from these results that both conditioning of hydraulic conductivity fields and sampling of soil moisture values through time produce comparable results and an improvement over the unconditioned-unassimilated case. The conditioned hydraulic conductivity field results were produced with a 10-member ensemble and for this case, require a small computational expenditure; however, an intensive one-time field and laboratory campaign would be necessary to generate these results in a real forecasting situation. Assimilating soil moisture observations using the EnKF/EAKF technique requires a larger 120-member ensemble to yield the results shown here, which requires greater computational resources, but collecting field observations would be relatively inexpensive, requiring only a single soil moisture probe installed near the surface. The economics of each individual application of either of these systems would determine which technique would be best suited for the specific application.

Figure 10.

A comparison of three different data assimilation schemes. Ensemble average, synthetic truth, and 95% confidence intervals for winds in region EAST are shown in Figures 10a, 10c, and 10e, and average ensemble spread and average ensemble mean RMSE are shown in Figures 10b, 10d, and 10f. A simulation with a subsurface conditioned with 200 hydraulic conductivity values produced Figures 10a and 10b (case COND200), assimilation of soil moisture values from one surface location in Figures 10c and 10d (case ONE-SMC/120), and both a conditioned subsurface and assimilation of soil moisture values from a single surface location are shown in Figures 10e and 10f (case COND200/ONE-SMC). Wind values are averaged over region EAST.

4. Conclusions

[34] We have developed an ensemble-based data assimilation system to incorporate observations into a fully coupled hydrologic and atmospheric modeling system. Such a system has not been applied to coupled hydrology-land-atmosphere processes before. The primary goal of this study is to verify the functionality of the new PF.WRF-DART forecast simulation system, to reveal its strengths and limitations, and to outline a course of further study to validate the system and evaluate it for potential operational implementation. We have demonstrated that the WRF-DART interface can be extended to include land surface and subsurface processes in the EnKF state vector and that observations at the interface can and do impact the forecast error covariance matrix of the model such that we can assimilate soil moisture observations into the coupled system and observe sensitivity to uncertainty in soil moisture distribution.

[35] Using the EnKF comes with a computational cost beyond the parameter adjustment work shown in Williams and Maxwell [2011]. In order to effectively accomplish the assimilation of observations using the EnKF system, large ensembles are required. The test cases in this study showed strongest results with 120 ensemble members, representing a 12-fold increase in computational demand over the simple parameter adjustment. With an ensemble of insufficient size, soil moisture results show a negative bias in the test cases performed. The ensemble adjustment Kalman filter of Anderson [2001] can partially solve the bias and excessive error problem and reduce the number of ensemble members needed, but still a large ensemble is most effective. Another potential source of the bias in the results may lie in the static parameterization of hydraulic conductivity fields. In its current configuration, the hydraulic conductivity fields are fixed, and even as more information is added to the system via assimilation of soil moisture observations, these fields are not evaluated or updated through the data assimilation model run. ParFlow uses Van Genuchten's [1980] empirical relationship between pressure head and soil moisture, and in this relationship soil moisture is dependent on hydraulic conductivity. Increasing soil moisture observations will increase the dimensions of the observation error covariance matrix and the dimensions of the Kalman gain, but the static hydraulic conductivity fields keep the degrees of freedom in the model state calculation restricted. This may serve to restrain the evolution of the model state in response to transient fields whose evolution could provide valuable information for subsurface characterization. This effect is amplified when hydraulic conductivity random fields are conditioned with measured values further restricting degrees of freedom. This limitation may need to be addressed to realize the full utility of the PF.WRF-DART system, a potential avenue for further work in developing this system.

[36] In spite of the bias revealed in the soil moisture results, the PF.WRF-DART system shows compelling results for uncertainty reduction in wind speeds measured downwind of soil moisture observation locations. With just a single soil moisture observation, the system was able to improve the spread-skill relationship for wind speeds downwind to a similar or better extent than by conditioning the subsurface hydraulic conductivity parameter alone. A full validation study incorporating atmospheric observations in addition to the soil moisture observations discussed here could demonstrate the full potential of this system and could serve to optimize field data collection strategies to support its use in an operational setting.

Acknowledgments

[37] This work was funded by the Center for Research and Education in Wind. The Golden Energy Computing Organization at the Colorado School of Mines provided computational resources used in this project. The authors acknowledge the helpful comments of H. J. Hendricks-Franssen and S. Kollet during the preparation of this manuscript.

Ancillary