Estimating surface turbulent heat fluxes from land surface temperature and soil moisture observations using the particle batch smoother

Authors


Abstract

Surface heat fluxes interact with the overlying atmosphere and play a crucial role in meteorology, hydrology, and climate change studies, but in situ observations are costly and difficult. It has been demonstrated that surface heat fluxes can be estimated from assimilation of land surface temperature (LST). One approach is to estimate a neutral bulk heat transfer coefficient (CHN) to scale the sum of turbulent heat fluxes, and an evaporative fraction (EF) that represents the partitioning between fluxes. Here the newly developed particle batch smoother (PBS) is implemented. The PBS makes no assumptions about the prior distributions and is therefore well-suited for non-Gaussian processes. It is also particularly advantageous for parameter estimation by tracking the entire prior distribution of parameters using Monte Carlo sampling. To improve the flux estimation on wet or densely vegetated surfaces, a simple soil moisture scheme is introduced to further constrain EF, and soil moisture observations are assimilated simultaneously. This methodology is implemented with the FIFE 1987 and 1988 data sets. Validation against observed fluxes indicates that assimilating LST using the PBS significantly improves the flux estimates at both daily and half-hourly timescales. When soil moisture is assimilated, the estimated EFs become more accurate, particularly when the surface heat flux partitioning is energy-limited. The feasibility of extending the methodology to use remote sensing observations is tested by limiting the number of LST observations. Results show that flux estimates are greatly improved after assimilating soil moisture, particularly when LST observations are sparse.

1 Introduction

The magnitude of available energy at the land surface and, particularly, its partitioning between sensible and latent heat fluxes directly affects land-atmosphere interactions and boundary layer formation [Bateni and Entekhabi, 2012]. Accurate estimates of surface turbulent heat fluxes are important in many fields such as meteorology, hydrology, and climate change studies [Caparrini et al., 2004a; Rigden and Salvucci, 2015].

In situ measurements of surface heat fluxes are difficult and expensive, and they are only available from a handful of sparse flux tower networks (e.g., FluxNet, AmeriFlux, and AsiaFlux) [Baldocchi et al., 2001]. Mapping regional heat fluxes from point measurements is hampered by strong spatial heterogeneity [Semmens et al., 2015], which is influenced by factors such as vegetation cover, soil moisture, and local landscape [Caparrini et al., 2004a]. Remote sensing techniques provide measurements of the land surface at various spatial and temporal scales and are a promising data source for surface heat flux estimation. Unfortunately, surface heat fluxes do not have a unique signature that can be detected directly by remote sensing instruments [Sini et al., 2008].

In previous studies, surface heat fluxes have mainly been estimated using four groups of methods. In the first group, empirical relationships are built between heat fluxes and local predictors such as land surface temperature (LST) and vegetation indices (VI) [Gillies et al., 1997; Nagler et al., 2005a, 2005b; Tang et al., 2010]. This group of methods is also referred to as the “triangle method” [Carlson, 2007]. In the second group, surface heat fluxes are estimated by solving the surface energy balance (SEB) [Kustas et al., 1996; Anderson et al., 1997; Bastiaanssen et al., 1998a, 1998b; Jiang and Islam, 2001; Su, 2002; Timmermans et al., 2007; Allen et al., 2007; Anderson et al., 2011]. The SEB models often utilize instantaneous LST observations to solve the instantaneous energy balance [Kalma et al., 2008]. Ancillary data such as surface roughness and leaf area index (LAI) are needed, and a closure assumption is often imposed [Sini et al., 2008]. A major assumption is that ground heat flux is a fraction of net radiation.

In a departure from the diagnostic methods, the third group estimates surface heat fluxes by assimilating data into land surface models (LSM) [Oleson et al., 2010; Niu et al., 2011] to update land surface states and parameters [Yang et al., 2007; Li et al., 2012; Sawada and Koike, 2014; Han et al., 2014]. Data assimilation is the technique of combining complementary information from model simulation and observations into an optimal estimate of the geophysical field of interest [Reichle, 2008]. The most commonly used methods are variational methods, ensemble methods, and particle methods. Variational data assimilation (VDA) methods merge model simulation with observations by constructing and minimizing a cost function derived from the forward model within a time window [Alavi et al., 2009]. A VDA method can be one-dimensional (1-D-Var), three-dimensional (3-D-Var), or four-dimensional (4-D-Var) depending on the spatial and temporal dimensions of the states. Ensemble methods are based on the Monte Carlo theory and estimate state variables by propagating and updating an ensemble of model states. They can be subdivided into sequential filtering (e.g., ensemble Kalman filter (EnKF)) [Reichle et al., 2010, 2013] and smoothing methods (e.g., ensemble Kalman smoother (EnKS)) [Dunne and Entekhabi, 2005, 2006; Dunne et al., 2007; Bateni and Entekhabi, 2012]. Particle methods have their origin in Bayesian estimation, and model states are estimated as the weighted sum of all particle estimates, where the particle weights are derived from a likelihood function [Moradkhani et al., 2012; Yan et al., 2015].

In the fourth group, LST time series are assimilated into heat transfer models to estimate two key parameters which characterize surface heat fluxes: neutral bulk heat transfer coefficient (CHN) and evaporative fraction (EF). CHN scales the sum of available energy, while EF governs the partitioning between sensible and latent heat fluxes. Sensible heat flux is calculated using CHN and meteorological data, and latent heat flux is estimated using sensible heat flux and EF. In consequence of the simple model structure, this group of methods greatly reduces the data demand compared to using LSMs. This group of methods has been applied using variational [Castelli et al., 1999; Boni et al., 2001; Caparrini et al., 2003, 2004a, 2004b; Sini et al., 2008; Bateni and Liang, 2012; Bateni et al., 2013a, 2013b; Xu et al., 2014, 2015] or ensemble smoothing [Bateni and Entekhabi, 2012] methods. This study belongs to the fourth group, and a detailed description of the method can be found in section 2.4.

The PBS has recently been introduced separately by Margulis et al. [2015] to estimate snow water equivalent and by Dong et al. [2015] to estimate soil moisture from distributed temperature sensing. It can be seen as an extension of the particle filter (PF) [Dong et al., 2015] as both algorithms use similar marginal distribution to derive the particle weights in the updating process. The difference is that states and parameters within a window are updated in a batch using all available observations in that window in the PBS, while the PF assimilates observations sequentially. As the dimension of states of the PBS is usually higher than that of the PF, the PBS requires a large amount of particles to avoid particle degeneracy. Despite the computational demand, the PBS is unique in many aspects.

  1. Compared to the variational methods [Caparrini et al., 2003, 2004a, 2004b; Sini et al., 2008; Bateni and Liang, 2012; Bateni et al., 2013a, 2013b], the PBS requires no computation of model adjoint or background error covariance; hence, it is much easier to implement.
  2. Compared to the ensemble (e.g., EnKF and EnKS) methods [Bateni and Entekhabi, 2012], the PBS makes no assumptions about the prior distributions, which is theoretically more accurate for hydrological applications in which the prior distributions are often non-Gaussian (a demonstration of the prior distribution of model states from this study is provided in Supporting Information) and the performance of ensemble methods are often suboptimal [Moradkhani et al., 2005; Dong et al., 2015; Yan and Moradkhani, 2016]. It is also better suited to parameter estimation [Dong et al., 2016a], as the PBS tracks the entire prior distribution of parameters using Monte Carlo sampling, which performs more robustly when the Gaussian error assumption is violated [DeChant and Moradkhani, 2012].
  3. Compared to the PF, the PBS utilizes information contained not only in every single observation but also in the temporal evolution of observations, as all available observations in the window are assimilated in a batch. This makes the PBS preferable in estimating surface heat fluxes from LST time series.

In this study, the PBS is used to estimate surface heat fluxes by assimilating LST observations into a heat transfer model through a joint state-parameter assimilation strategy in the first experiment (section 2.4.1). This is the first study that adopts the PBS to estimate surface heat fluxes by assimilating LST data.

Although LST time series contains information about surface energy partitioning, many studies have demonstrated that the assimilation strategy performs poorly on wet or densely vegetated surfaces [Caparrini et al., 2004a; Crow and Kustas, 2005; Bateni and Entekhabi, 2012; Xu et al., 2014]. This happens because under these conditions, the surface energy partitioning becomes more energy-limited, which weakens the constraint of LST on surface energy balance [Caparrini et al., 2004a]. Sini et al. [2008] demonstrated that using soil wetness information to constrain EF could improve flux estimation under these conditions. Soil moisture controls the partitioning of available energy into sensible and latent heat fluxes through its influence on evapotranspiration [Entekhabi et al., 1996; Margulis et al., 2002; Koster et al., 2004; Entekhabi et al., 2010; Seneviratne et al., 2010; Crow et al., 2015]. Many studies have demonstrated a positive correlation between EF and soil moisture at different depths [Kustas et al., 1993; Lhomme and Elguero, 1999; Dirmeyer et al., 2000; Basara and Crawford, 2002; Wang et al., 2006; Gentine et al., 2007; Santanello et al., 2011]. Here we investigate for the first time in depth the potential value of joint soil moisture and LST assimilation through comparative experiments. In the second experiment (section 2.4.2), a simple soil water transfer scheme is introduced and coupled to the heat transfer model, and soil moisture observations are assimilated simultaneously with LST observations. To provide an additional constraint on EF, a soil wetness-EF relationship is adopted.

A potentially interesting extension of this study is to use LST and soil moisture observations from remote sensing. This would open the path to estimating surface heat fluxes consistently at larger scales. Soil moisture products are available from L-band microwave remote sensing from missions such as the Soil Moisture and Ocean Salinity (SMOS) mission and the Soil Moisture Active Passive (SMAP) mission, which provide global soil moisture observations every 2–3 days [Kerr et al., 2001; Entekhabi et al., 2010]. However, the estimation robustness may be affected by the number of available LST observations, in addition to the influence of spatial resolution and data accuracy, among others. Potential sources for LST observations include the Advanced Very High Resolution Radiometer (AVHRR), the Moderate Resolution Imaging Spectroradiometer (MODIS), and the Geostationary Operational Environmental Satellites (GOES), among others. Typically, the same area is observed no more than twice each day by polar-orbiting satellites, and the observations may fall outside the nominal assimilation window. For geostationary satellites, cloudy-sky conditions which represent more than half of the day-to-day weather [Jin, 2000] can dramatically reduce the number of available observations. A simulation test is conducted to assess the influence of LST data availability on flux estimates.

Based on the discussion above, there are three objectives of this study: (1) to investigate the performance of the PBS on the assimilation of LST observations for surface heat flux estimation; (2) to introduce a soil moisture transfer scheme to constrain EF and jointly assimilate LST and soil moisture observations to improve the poor performance on wet and densely vegetated surfaces; (3) to explore the influence of LST data availability on flux estimation in the potential application with remote sensing data. We aim to answer the following three science questions: (1) What are the effects of assimilating LST observations to estimate surface heat fluxes with the PBS? (2) Can the estimation be improved if soil moisture observations are assimilated simultaneously with LST observations, particularly on wet or densely vegetated surfaces? (3) Given the data availability issue in potential applications with remote sensing data, will the conclusions hold when the number of LST observations is limited?

This paper is organized as follows: Section 2 describes the PBS algorithm, the models used, the experiment design and the data sets. Section 3 provides the assimilation results and the discussion. Finally, the conclusions are drawn in section 4.

2 Materials and Methods

2.1 Study Area and Data Sets

The experiments are conducted using data from the First ISLSCP (International Satellite Land Surface Climatology Project) Field Experiment (FIFE) which took place in the summers of 1987 and 1988 in the prairies in central Kansas [Sellers et al., 1992]. During FIFE, meteorological data were routinely measured with Portable Automatic Meteorological (PAM) stations. LST was measured with a downward looking radiometer at each PAM station, and surface fluxes were measured at 22 and 10 sites in 1987 and 1988 using either Bowen ratio or eddy-covariance instruments. Considering the data quality and data sampling problems at individual sites [Duan et al., 1996], the site-averaged data sets of half-hourly forcing data, LST, and surface flux observations provided by Betts and Ball [1998] are used here. This study is comparable to the previous studies using the same data sets [Caparrini et al., 2004a; Crow and Kustas, 2005; Bateni and Entekhabi, 2012; Bateni and Liang, 2012; Bateni et al., 2013a].

Soil moisture was systematically measured using the gravimetric method for the top 10 cm soil and neutron probes to a depth of up to 2 m. To generate a FIFE average, Betts and Ball [1998] first averaged measurements at each site, then obtained one daily value for each site, and after that, averaged these values to get a daily FIFE site average. The soil moisture values for the 0–5 cm soil are used, and the gravimetric values are converted to volumetric soil moisture by multiplying a bulk soil density of 1.1 math formula as suggested by Betts and Ball [1998]. Soil texture falls in the texture classes of silty clay and silty clay loam, and the bulk densities range from 0.96 to 1.5 math formula.

The longest contiguous periods during the FIFE experiment (day 148–243 for FIFE 87 and 160–243 for FIFE 88) are used for assimilation and validation. To facilitate intercomparison, the same time periods are adopted as used in previous studies [Caparrini et al., 2004a; Bateni and Liang, 2012; Bateni and Entekhabi, 2012; Bateni et al., 2013a]. Figure 1 shows the time series of the half-hourly precipitation data and daily-averaged volumetric soil moisture of the top 5 cm soil of FIFE 87 and 88. The data from the two campaigns are very different. FIFE 87 experienced a very wet initial period and a long dry down (until day 166). After that, soil moisture fluctuated with precipitation, and another long dry down ran from day 187 to day 211. In contrast, during FIFE 88, the soil was in general much drier, and soil moisture responded more slowly to precipitation compared to FIFE 87. It should be noted that soil moisture was sampled less frequently during FIFE 88 [Betts and Ball, 1998], which may reduce the accuracy of the site-average data.

Figure 1.

Half-hourly precipitation and daily soil moisture (0–5 cm) time series of (a) FIFE 87 and (b) FIFE 88.

2.2 Surface Energy Balance

The flux estimation theory is fundamentally based on the surface energy balance equation

display math(1)

where Rn is net radiation, H is sensible heat flux, LE is latent heat flux, and G is ground heat flux. H can be calculated from the vertical gradient of temperature between the land surface and the near-surface air by

display math(2)

Here ρ [kg/m3] is air density, Cp [J/kg/K] is specific heat capacity of air, CH is the bulk coefficient for heat transfer, U [m/s] is wind speed, Ts [K], and Tair [K] are the temperature of land surface and near-surface air.

CH is mainly dependent on two factors: the landscape characteristics and the atmospheric stability. As the influence of landscape depends mainly on the surface geometry and the vegetation phenology, it varies slowly over time (e.g., monthly) [Caparrini et al., 2003, 2004a, 2004b; Crow and Kustas, 2005; Sini et al., 2008; Bateni and Liang, 2012; Bateni and Entekhabi, 2012; Bateni et al., 2013a, 2013b; Xu et al., 2014, 2015]. Here we adopt the stability correction function introduced by Caparrini et al. [2003] to estimate CH. The stability correction function has proved effective in several studies to estimate surface heat fluxes [Caparrini et al., 2003, 2004a, 2004b; Crow and Kustas, 2005; Sini et al., 2008; Bateni and Liang, 2012; Bateni and Entekhabi, 2012; Bateni et al., 2013a, 2013b; Farhadi et al., 2014; Xu et al., 2014, 2015]. The function is given by

display math(3)

where CHN is the CH under neutral atmospheric condition, which represents the influence of land surface characteristics on surface heat fluxes, and Ri is the Richardson number which is an indicator of the atmospheric stability. Ri is estimated by

display math(4)

where g [m/s2] is gravitational acceleration, Tpot [K] is potential temperature, z [m] is vertical height, and Δ represents the difference across height difference math formula. Ri is dependent on atmospheric conditions and exhibits strong diurnal variation. When the atmosphere is unstable, math formula is negative, which leads to a negative Ri, and vice versa. H can be calculated if CHN and Ri are determined.

The EF is introduced to calculate LE. EF is defined as

display math(5)

which renders

display math(6)

The primary benefit of using EF is that it is almost constant for near-peak radiation hours (09:00–16:00) on days without precipitation [Crago, 1996; Crago and Brutsaert, 1996; Gentine et al., 2007]. Therefore, using EF greatly reduces the number of parameters to be estimated, and increases the robustness of the retrieval in the data assimilation applications [Caparrini et al., 2004a].

2.3 The Particle Batch Smoother (PBS)

2.3.1 PBS Algorithm

Here the PBS formulations as outlined by Dong et al. [2015] are used. The update process of the PBS is similar to that of the PF. Initially, all particles are given the same weight. When observations are assimilated, the updated weights are calculated using the prior distribution and likelihood of each particle. Consequently, particles closer to the observations get larger weights.

The evolution of model states in time can be described by

display math(7)

where f is the forward model, math formula is the model state (T and θ in this study) vector of the ith particle at time step t, math formula is the perturbed forcing data, math formula is the model parameter vector, and math formula represents model error. Here math formula is assumed to be normally distributed.

The model states are related to the observations by

display math(8)

where math formula is the simulated observation vector, h is the observation operator that translates modeled states to the observations, and math formula is the observation error.

For an assimilation window of length L, the updated particle weights are derived by

display math(9)
display math(10)

where math formula is the weight of the ith particle for the entire assimilation window, math formula is the unnormalized weight from importance sampling, and math formula is the likelihood function, which is expressed as

display math(11)

Here R is the error covariance matrix of observations, and math formula is the observation vector. The reader is referred to Dong et al. [2015] for a detailed derivation of the PBS.

2.3.2 The Tuning Factor

Particle degeneracy, which is the situation when most of the particles have negligible weights, can severely weaken the performance of the PBS. To avoid this problem, resampling of the posterior after each update is performed [Moradkhani et al., 2005]. However, for cases when the observations are nearly perfect (i.e., very small R) or the model estimations are very inaccurate (e.g., in consequence of bad initialization), resampling alone cannot prevent particle degeneracy [Stordal et al., 2011]. For such cases, the variance of particle weights will be extremely high, giving too much importance to a few particles, and most of the particles will be removed after resampling. As a result, the estimates will be unreliable. This problem also occurs when the dimension of model states is high [Bengtsson et al., 2008]. Stordal et al. [2011] suggested that this problem could be avoided by approximating the posterior with heavy tails. Although biases are introduced in this process, the final estimates are almost surely to converge to the true posterior.

Dong et al. [2016a] introduced a tuning factor of β that modified the likelihood function (equation (11)) as

display math(12)

where β ranges from 0 to 1, effectively reducing the variance of the particle weights after updates. Small β values essentially allow the particle spread to be wide enough to encompass the observations within the PBS assimilation window. When β = 1 is used, the modified likelihood function is reduced to that in equation (11). The optimal value of β depends on the specific application. The model is run for a number of reasonable β values, and the β value that minimizes the RMSE of flux estimates is chosen as the optimal value. In this study, the optimal β values are 0.8 and 0.5 for the first and the second experiment.

2.4 Experimental Setup

Two experiments were conducted in this study. In the first experiment (section 2.4.1), the PBS was used to assimilate LST into the force-restore model to estimate surface fluxes (hereafter PBST). In the second experiment (section 2.4.2), a soil moisture transfer scheme was introduced, and LST and soil moisture data were assimilated simultaneously using the PBS (hereafter math formula). Off-line convergence tests showed that there was only marginal improvement in the RMSE of flux estimates when over 100 particles were used in both experiments. As additional particles were not computationally expensive, to ensure sufficient particles for state and parameter estimation, 300 particles were used in both experiments. Following the approaches of Caparrini et al. [2003, 2004a], Sini et al. [2008], Bateni and Entekhabi [2012], Bateni and Liang [2012], and Bateni et al. [2013a], the PBS was implemented using a daily assimilation window from 09:00 to 16:00, during which the EF can reasonably be assumed to be a constant. At the beginning of each time period, the CHN value for each particle was randomly sampled within a given range. Table 1 shows the valid ranges for CHN initialization in each time period. The ranges were determined by the “optimal values” provided by Caparrini et al. [2004a] plus a ±1 × 10−3 variation. Caparrini et al. [2004a] determined these “optimal values” by minimizing the cost function through a variational scheme. The CHN ranges imply considerable uncertainty while maintaining the validity of CHN within a 30 day time period.

Table 1. Initial Range of CHN for Each Time Perioda
FIFE 87FIFE 88
DayOptimal CHNCHN RangeDayOptimal CHNCHN Range
  1. a

    The Optimal CHN Values Were Derived by Caparrini et al. [2004a].

148–1778.96 × 10−3(7.96–9.96) × 10−3160–1901.91 × 10−3(0.91–2.91) × 10−3
178–2067.12 × 10−3(6.12–8.12) × 10−3191–2203.76 × 10−3(2.76–4.76) × 10−3
207–2434.60 × 10−3(3.60–5.60) × 10−3221–2433.59 × 10−3(2.59–4.59) × 10−3

2.4.1 PBS With Only LST

In this experiment, the force-restore model was used to give the time evolution of LST in response to atmospheric forcing and the restoring effect of the deep soil

display math(13)

Here Pe math formula is the effective thermal inertia, ω math formula is the diurnal frequency, Ts [K] and TD [K] are LST and deep soil temperature, and ϵ represents model error. Pe is given a constant value of 750 math formula following Caparrini et al. [2004a] and Sini et al. [2008]. It was shown that variations in Pe did not significantly affect the results [Sini et al., 2008]. TD is estimated with a semidiurnal filter of the land surface following Caparrini et al. [2003]. An additive Gaussian error with a standard deviation of 0.1 K is added at each time step.

The soil texture particles were randomly sampled within the two texture classes and bulk density values were sampled from the range shown in Table 2. The corresponding hydraulic properties were generated from ROSETTA software [Schaap et al., 2001]. The forcing data in this experiment are net radiation, air temperature, and wind speed. Different forms of perturbations have been applied to characterize the error distributions of forcing data [Reichle et al., 2008; Leisenring and Moradkhani, 2011; Dong et al., 2016a]. In this experiment, the forcing data perturbations are described in Table 2 following Bateni and Entekhabi [2012] and Dong et al. [2016a]. To make the perturbation less subjective, approaches such as the variable variance multiplier can be used to dynamically adjust the ensemble spread of the state and parameter predictions in future studies [Leisenring and Moradkhani, 2012].

Table 2. Perturbation of Forcing Data and Soil Properties
VariablePerturbationStandard DeviationBound
Silt [%]Uniform USDA texture class
Clay [%]Uniform USDA texture class
Bulk Density [ math formula]Uniform [0.96,1.5]
Net Radiation [W/m2]Gaussian, ×0.1 × Net radiation 
Air Temperature [K]Gaussian, +1 
Wind Speed [m/s]Gaussian, +0.1 
Precipitation [mm/s]Gaussian, ×0.1 × Precipitation 

The model was run from 09:00 to 16:00 local time at half-hourly time steps (i.e., 15 time steps per day). Each day at 09:00, the LST particles need to be initialized. Here the first available observations, typically at 09:00, were used to provide an initial condition for the assimilation window, and a 3 K additive Gaussian error was used following Bateni and Entekhabi [2012]. If the in situ observations are not available, the particles can also be initialized with data from other sources, such as geostationary satellites, land data assimilation systems, and reanalysis data. For each particle, a daily average EF was randomly sampled from a uniform distribution with a range of 0.1–0.9. At every time step, H was first calculated from equation (2), and LE was derived from H and EF using equation (6). The force-restore model was then used to propagate LST to the next time step.

At 16:00, all available LST observations (14 observations in this experiment) of the day were assimilated using the PBS. The state vector for particle i is

display math(14)

Here math formula are the time steps when LST observations are available. Particle weights are determined from equation (10). The flux estimates at each time step were calculated as a weighted sum of all particles, and CHN particles were resampled to give the prior estimates for the next day. Results are compared to an open-loop (OL, i.e., no assimilation case) run (Figures 2-6).

Figure 2.

Estimated daily (09:00–16:00) average LST from PBST and OL versus observations for (a) FIFE 87 and (b) FIFE 88 with 14 LST observations assimilated.

Figure 3.

Estimated daily (09:00–16:00) average EF from PBST and OL versus observations for (a) FIFE 87 and (b) FIFE 88 with 14 LST observations assimilated. Daily soil moisture data are plotted in dashed line.

Figure 4.

CHN estimates from PBST and math formula for (a) FIFE 87 and (b) FIFE 88 with 14 LST observations assimilated.

Figure 5.

Scatterplot of daily (09:00–16:00) average modeled fluxes versus observations for (a–f) FIFE 87 and (g–l) FIFE 88 from OL (black), PBST (red), and math formula (blue) with 14 LST observations assimilated. The units of RMSE and Bias are W/m2. The results for the three time periods are plotted separately.

Figure 6.

Scatterplot of half-hourly (09:00–16:00) modeled fluxes versus observations for (a–f) FIFE 87 and (g–l) FIFE 88 from OL (black) and PBST (red) with 14 LST observations assimilated. The units of RMSE and Bias are W/m2. The estimates at 09:00 are plotted in large open circles and the other estimates in small dots. The results for the three time periods are plotted separately.

2.4.2 PBS With Both LST and θ

In the second experiment, a simple soil moisture transfer scheme was introduced. The aim is not to model soil moisture transfer accurately but to further constrain the EF and flux estimation. The scheme used is taken from the Simple Biosphere (SiB) model [Sellers et al., 1986]. For N soil layers, the soil moisture variation of each layer is calculated by

display math(15)

Here Wk is the soil wetness of the kth layer, θs math formula is the volumetric soil moisture at saturation, I1 [cm/s] is the infiltration into the first layer from precipitation, Dk [cm] is the thickness of the kth layer, math formula [cm/s] is the flow between the kth and k + 1th layer, ρw [g/cm3] is the water density, Es math formula is the water loss from soil evaporation, math formula math formula is the water loss from vegetation transpiration in the kth layer, and QN [cm/s] is the gravitational drainage from the deepest layer. The additive modeling error was assumed Gaussian with a standard deviation of 0.001 math formula. The scheme was run at a 30 min time step, and runoff was neglected.

Wk is defined as

display math(16)

where θk math formula is the soil moisture of the kth layer and θr math formula is the residual moisture.

I1 is given by

display math(17)

where Prate [cm/s] and Ksat [cm/s] are precipitation rate and the hydraulic conductivity at saturation.

Following the formulation in Sellers et al. [1986], the water transfer between adjacent layers is given by

display math(18)

where K [cm/s] and ψ [cm] are hydraulic conductivity and soil moisture potential, which are derived using the Van Genuchten [1980] method.

To implement the scheme, the estimated LE needs to be partitioned into soil evaporation Es and vegetation transpiration Et. The partitioning is conducted by assuming

display math(19)

The LAI values in each time period can be found in Bateni and Entekhabi [2012]. These LAI values were derived from the LAI-NDVI (Normalized Difference Vegetation Index) relationship [Aparicio et al., 2000] and the site average NDVI data from Landsat and SPOT satellites [Hall et al., 1992]. The constant c governs the radiation extinction on the basis of the canopy of sun angle, plant distribution, and the arrangement of leaves [Simunek et al., 2005]. Values suggested for c vary from −0.82 [Campbell, 1985] to −0.5 [Kustas et al., 1996; Anderson et al., 1997] and −0.463 in HYDRUS-1D [Simunek et al., 2005; Sutanto et al., 2012]. Here c was set to −0.5. Sensitivity tests demonstrated that small variations of c did not affect flux estimates significantly.

The transpiration water loss from each layer is derived from the overall percentage of available moisture ( math formula) weighted by root fraction

display math(20)

where math formula is the root fraction of the kth layer. The root distribution function adopted is the same as that used in the Community Land Model (CLM) model

display math(21)

where Y is the cumulative root fraction from the surface, and d is soil depth. a and b are empirical parameters for different vegetation types. Values of 10.74 and 2.608 are assumed for a and b, as suggested by Zeng [2001].

The modeled soil moisture data are used in two ways: to constrain daily EF and to serve as a model state in the assimilation. Here the relationship proposed by Dirmeyer et al. [2000] is adopted as

display math(22)

where EFref is the prior guess of the reference daily EF, EFmax is the maximum possible EF and can be safely assumed to be unity, α is a slope factor that controls the shape of the curve, and SWI is the soil wetness index calculated from field capacity and wilting point based on soil texture. Here SWI was taken as the mean of all particle estimates.

At 09:00 each day, the SWI of each particle was calculated, and a daily reference EFref was generated using equation (22). The daily average EF of each particle was uniformly sampled within the range of EFref ± ϵ. In this experiment, α = 4 was used which proves effective to capture the arctangent EF-SWI relationship, and ϵ = 0.2 was shown to allow a reasonably large and valid range of EF variation.

From 09:00 to 16:00, fluxes are estimated using equations (2) and (6), and precipitation data and LE estimates are used to force the water transfer scheme to propagate soil moisture. From 16:00 to 09:00 the next day, LE cannot be derived as EF is no longer conservative. Here G is estimated randomly as a fraction of Rn [Choudhury et al., 1987; French et al., 2003], and LE is calculated as the residual of surface energy balance:

display math(23)

where cG is a coefficient indicating the highest percentage of G in Rn, and math formula represents a random number between 0 and 1. Based on the statistics in Betts and Ball [1998], cG was assumed to be 0.3. In general, fluxes between 16:00 and 09:00 were relatively low, and test showed that this simple scheme worked reasonably well for flux estimation.

In this experiment, the soil column was divided into six layers. The layer thicknesses were 5, 10, 15, 15, 15, and 30 cm, respectively. In contrast to PBST which requires state initialization every day, LST and θ only need to be initialized once at the beginning of the experiment in math formula. The model was then propagated with perturbed forcing data shown in Table 2. At the end of the day, the available LST and 0–5 cm θ observations were assimilated. We only assimilated θ of the top 5 cm, which corresponds to the typical penetration depth of L-band microwave remote sensing observations, such as soil moisture products from SMOS and SMAP. As only daily average θ was available, the mean modeled θ was calculated as the prior estimate. The state vector for particle i is

display math(24)

where math formula are the time steps when LST observations are available and math formula is the soil moisture of the top layer of the ith particle at time step j. The observation errors for θ was assumed 0.04 math formula, which is typical for remote sensing observations [Das et al., 2011; Jackson et al., 2012]. After assimilation, LST and θ as well as CHN were resampled and acted as the prior estimates for the next day.

To assess the impact of LST data availability on flux estimation, the model was run multiple times with different number of available LST observations (1, 2, 3, 4, 5, 10, and 14, respectively) for both assimilation strategies, and the corresponding mean RMSEs were calculated. The time of observations was randomly chosen in the assimilation window to simulate the random occurrence of cloud contamination.

3 Results and Discussion

3.1 PBS With Only LST

Figure 2 shows the estimated daily (09:00–16:00) average LST from PBST and open-loop (OL) versus observations. The three time periods of each campaign are delineated with dash-dotted lines. It is evident that the PBST captures the temporal trend of LST very well and is always closer to the observations than the OL for both FIFE 87 and FIFE 88. The RMSE decreases from 3.00 K (OL) to 0.81 K (PBST) for FIFE 87, and from 4.01 K (OL) to 0.53 K (PBST) for FIFE 88. The improvement is more evident for FIFE 88 than for FIFE 87. As the soil is generally much drier in FIFE 88, the surface energy partitioning is more moisture-limited. This makes LST a stronger constraint on surface heat flux partitioning for FIFE 88 than for FIFE 87.

Figure 3 shows the estimated daily (09:00–16:00) average EF from the PBST and OL versus EF calculated from observed turbulent heat fluxes. The three time periods of each campaign are delineated with dash-dotted lines. As EF is not constrained in OL runs, the estimated daily average EFs stay almost constant and deviate a lot from the observations, especially for days when the actual EF is high. In the PBST, the estimated EFs capture the daily variations in observed EFs, and the estimated EF values are generally much closer to the observations.

Daily soil moisture observations are also plotted in Figure 3 to facilitate analysis. PBST performs less satisfactorily for days when the soil is very wet (e.g., day 148–177 in FIFE 87) and more accurately for dry-down events (e.g., day 200–210 and 230–243 in FIFE 88). For wet soil, the soil moisture is high enough to supply unlimited water for evaporation and transpiration, and the surface energy partitioning is controlled mainly by surface properties and atmospheric conditions [Shokri et al., 2008; Bateni and Entekhabi, 2012]. Therefore, the coupling between EF and LST becomes very weak, and the estimation of EF from LST is very uncertain. In contrast, during a dry-down event, EF is mainly controlled by soil moisture availability, leaving plenty of information of energy partitioning in LST time series, thus the estimation of EF is more accurate and robust. It is noted that for day 199–206 in FIFE 87, the EF estimates capture the falling trend but quickly drift away from the observations despite the dry-down event. This is caused by a sharp drop in CHN between time periods. A mean CHN of 7.12 × 10−3 is initially used for day 177–206, but CHN quickly drops to 4.31 × 10−3 for day 192–221 according to Caparrini et al. [2004a]. This may be caused by the changing vegetation phenology. According to Hall et al. [1992], the LAI quickly fell from about 1.5 to about 0.5 during day 180–215 in FIFE 87, which may explain to some extent the dramatic fall of CHN. As is shown in the estimated CHN time series in Figure 4, the PBS takes longer to converge to the much lower CHN value, leading to higher H and lower LE estimates, therefore EFs are underestimated. When CHN is initialized reasonably well for the third time period, EF is estimated more accurately (day 207).

Figure 5 shows the daily (09:00–16:00) average modeled H and LE versus observations for FIFE 87 and FIFE 88. Generally, significant improvement is seen for all time periods in both campaigns in terms of RMSE and R. The overall RMSEs decrease from 100.7 to 70.7 W/m2 for H, and from 101.3 to 72.7 W/m2 for LE in FIFE 87. In FIFE 88, the overall RMSEs decrease from 63.5 to 39.6 W/m2 for H, and from 103.4 to 56.1 W/m2 for LE. The statistics are comparable to the results reported in Caparrini et al. [2004a] which also used the force-restore model and variational assimilation. Besides, the improvement in estimation bias is also evident. Overall, the biases decrease from 86.4 to 51.2 W/m2 for H, and from −89.8 to −47.2 W/m2 for LE in FIFE 87. In FIFE 88, the overall biases decrease from 45.8 to 8.8 W/m2 for H, and from −89.9 to −29.7 W/m2 for LE.

Figure 6 shows the half-hourly flux estimates versus observations. The estimates at 09:00 are plotted as large open circles and the other estimates as small dots. This is to highlight the fact that some extremely large and negative results are obtained at 09:00, especially in LE estimates. This is caused by the weak hypothesis of constant EF at the very first time step of the assimilation window. In the early morning, LST is sometimes close to or even lower than the air temperature. After particle initialization at 09:00 each day, many particles will have values lower than air temperature. This leads to negative H estimates which are then magnified by the constant EF hypothesis in LE estimation. The same problem was also reported in Caparrini et al. [2004a]. The recalculated RMSE and R excluding estimates at 09:00 are shown in parenthesis. The improvement is also remarkable even at half-hourly timescale. When estimates at 09:00 are excluded, the overall RMSEs decrease from 116.9 to 83.6 W/m2 for H, and from 97.1 to 68.3 W/m2 for LE in FIFE 87. In FIFE 88, the overall RMSEs decrease from 75.9 to 46.3 W/m2 for H, and from 104.3 to 58.6 W/m2 for LE. The reduction in biases is also significant. In FIFE 87, the overall biases decrease from 97.4 to 60.6 W/m2 for H, and from −82.2 to −33.3 W/m2 for LE. In FIFE 88, the overall biases decrease from 52.3 to 12.3 W/m2 for H, and from −87.6 to −24.2 W/m2 for LE.

3.2 PBS With Both LST and θ

Figure 7 shows the 0–5 cm daily soil moisture estimates from OL and math formula versus observations. Although the subdaily dynamics are smoothed in the daily area average soil moisture data, improvement on soil moisture estimates after assimilation is still evident, particularly in FIFE 87. After assimilation, the soil moisture time series agrees better with observations. Despite the difference in absolute values, the PBS captures the dry-down events of day 148–166 in FIFE 87 and day 200–210 in FIFE 88, and the fluctuations during day 176–192 in FIFE 87 are captured very well. These improved soil moisture estimates would benefit the estimation of EFs.

Figure 7.

0–5 cm daily soil moisture estimates from OL and math formula versus observations for (a) FIFE 87 and (b) FIFE 88 with 14 LST observations assimilated.

A comparison of CHN estimates from PBST and math formula is shown in Figure 4. In general, CHN estimates from both strategies follow similar temporal trends. In FIFE 87, the CHN estimates from PBST are always higher than those from math formula, which explains the overall underestimation of EFs (Figure 3). In FIFE 88, the CHN estimates from both strategies are comparable, particularly in the last time period (day 221–243). This implies that LST time series is a strong constraint in surface energy partitioning, which is also demonstrated in the EF estimates shown in Figure 3. The temporal evolution of CHN can be explained by vegetation phenology. According to Hall et al. [1992], in FIFE 87 the LAI falls first until day 160, and then keeps rising until day 170–180 followed by a long fall until about day 215. After that, LAI gradually rises until day 243. The temporal evolution of LAI agrees very well with the CHN estimates from math formula. In FIFE 88, the CHN estimates vary only in a very small range in both strategies compared to FIFE 87. This may be explained by the small variation of LAI (1.2–1.4) during FIFE 88 reported in Bateni and Entekhabi [2012].

Figure 8 shows the estimated EFs from PBST and math formula. The shaded area indicates the prior guess of the EF range, and the blue and red lines are the estimates from math formula and PBST, respectively. Almost all observed EFs fall in the shaded area, showing the validity of the EF-SWI relationship. The math formula estimates are generally much closer to observations, demonstrating the benefit of also assimilating soil moisture observations. In FIFE 87, the improvement is most evident in the first time period (day 148–177) when the soil is very wet and for day 199–206 when CHN varies strongly. When the soil is wet enough to supply unlimited water, the LST is no longer a strong constraint on energy partitioning. Adding soil moisture assimilation improves EF estimation by forcing relatively high EF values. When CHN varies strongly, the soil moisture estimates prevent sharp fluctuations in EF. This ensures that CHN converges to more realistic values. In FIFE 88, the improvement is most evident in the first time period (day 160–190) which features many light rain events. When soil moisture assimilation is included, EF is allowed to vary within a more reasonable range, which has a great impact on EF and flux estimation.

Figure 8.

Estimated daily (09:00–16:00) average EF from math formula and PBST versus observations for (a) FIFE 87 and (b) FIFE 88 with 14 LST observations assimilated. The shaded area is the prior guess of valid EF range from math formula.

Daily (09:00–16:00) average flux modeled by math formula is plotted against observations in Figure 5. The most significant improvement over PBST is seen in the first two time periods (day 148–206) in FIFE 87 and the first time period (day 160–190) in FIFE 88 as expected. For other time periods where PBST results are already accurate, the benefit of including soil moisture is marginal. The overall RMSEs of H and LE are 26.7 and 41.7 W/m2 from math formula, compared to 70.7 and 72.7 W/m2 from PBST for FIFE 87. In FIFE 88, the overall RMSEs of H and LE are 30.9 and 38.2 W/m2 from math formula, compared to 39.6 and 56.1 W/m2 from PBST. Similarly, the estimation biases are to a large extent improved. The overall biases are reduced from 51.2 and −47.2 W/m2 for H and LE from PBST to 1.0 and 24.8 W/m2 from math formula in FIFE 87. In FIFE 88, the overall biases vary from 8.8 and −29.7 W/m2 for H and LE from PBST to −10.9 and −1.7 W/m2 from math formula. There is also a small increase in R compared to the results from the PBST.

Similar to Figure 6, the half-hourly fluxes from PBST and math formula are plotted against observations in Figure 9. One great advantage of math formula is that the problem of erroneous estimates at 09:00 is to a large extent solved. As LST particles are resampled at the end of the previous assimilation window and then modeled continuously during the night, the particle spread of modeled LST at 09:00 will be smaller compared to randomly generated particles from PBST. Therefore, it is less likely to result in large negative flux estimates. The decrease of RMSEs is evident when considering the full 09:00–16:00 period. When estimates at 09:00 are excluded, the improvement is more evident in H estimates. The RMSEs for H decrease from 83.6 to 33.7 W/m2 in FIFE 87 and from 46.3 to 38.5 W/m2 in FIFE 88. The RMSEs for LE decrease slightly from 68.3 to 63.8 W/m2 in FIFE 87, and from 58.6 to 52.9 W/m2 in FIFE 88. Reduction in estimation biases are also dramatic. The overall biases for H decrease from 60.6 to 1.3 W/m2 in FIFE 87 and from 12.3 to −10.6 W/m2 in FIFE 88. The overall biases for LE decrease from −33.3 to 27.2 W/m2 in FIFE 87 and from −24.2 to −0.7 W/m2 in FIFE 88.

Figure 9.

Scatterplot of half-hourly (09:00–16:00) modeled fluxes versus observations for (a–f) FIFE87 and (g–l) FIFE88 from PBST (red) and math formula (blue) with 14 LST observations assimilated. The units of RMSE and Bias are W/m2. The estimates at 09:00 are plotted in large open circles and the other estimates in small dots. The results for the three time periods are plotted separately.

The time series of estimated daily (09:00–16:00) average flux estimates from the OL, PBST and math formula are compared with the observations as well as Rn for FIFE 87 and FIFE 88 in Figure 10. The red and blue shaded areas indicate the particle spread (i.e., the range between minimum and maximum particle estimates) of PBST and math formula estimates. Compared to the results from PBST, the day-to-day variations of fluxes using math formula are more consistent with those of observations. The particle spread of both PBS methods varies with the magnitude of Rn. The particle spread gets larger when Rn is high (e.g., day 199–200 in FIFE 87), and smaller when Rn is low (e.g., day 224 in FIFE 87). Both PBS methods provide valid ranges of flux estimates, while the particle spread of math formula is smaller and more reasonable. Overall, the results show that the assimilation of LST time series with PBS can successfully estimate turbulent heat fluxes, and that introducing soil moisture constraints can improve the estimation, particularly when the surface energy partitioning is energy-limited.

Figure 10.

Estimated time series of daily (09:00–16:00) average H and LE from OL, PBST and math formula versus observations and Rn for (a, b) FIFE 87 and (c, d) FIFE 88 with 14 LST observations assimilated. The shaded areas are the particle spread of PBST (red) and math formula (blue).

Figure 11 shows the mean diurnal cycle of observed and estimated surface heat flux components for FIFE 87 and FIFE 88. In general, PBST overestimates H and underestimates LE in both campaigns, and this is also demonstrated by the underestimated EFs in Figure 8. math formula restores the diurnal behavior of surface heat fluxes better, particularly in FIFE 88. An evident deviation from the observations is seen at 09:00 using PBST as a result of the outliers at the very first time step of the assimilation window, especially in FIFE 87. When soil moisture is assimilated, this problem is largely solved.

Figure 11.

Diurnal cycle of average observed and estimated surface heat fluxes for FIFE 87 and FIFE 88 with 14 LST observations assimilated.

3.3 Influence of LST Availability

Results from the simulation test on LST data availability are shown in Figure 12. To eliminate the influence of negative outliers at 09:00, only estimates during 09:30 and 16:00 are used. At the daily (09:30–16:00) scale, the RMSEs of both H and LE decrease quickly with more available LST observations using PBST, particularly when the observations are sparse. When more than five LST observations are available, the benefit of more LST observations becomes less evident. This indicates that the diurnal variation of LST and surface heat fluxes can be restored reasonably well with as few as five LST observations using PBST. When soil moisture observations are assimilated, the flux estimates are greatly improved, particularly when LST observations are limited.

Figure 12.

Variation of the mean RMSE of flux estimates with available LST observations at daily (09:30–16:00) and half-hourly scales using PBST (red) and math formula (blue).

The half-hourly results from 09:30 to 16:00 are shown in Figure 12e through 12h. Similar to the daily results, the RMSEs decrease quickly with increasing number of LST observations at first, and stay almost constant when over five LST observations are available each day using PBST. When soil moisture observations are assimilated, the H estimates are significantly improved. When only one LST observation is available, the RMSEs can be reduced by as much as ∼70 (∼20) W/m2 for H and by ∼20 (∼20) W/m2 for LE in FIFE 87 (FIFE 88). This demonstrates that assimilating soil moisture information greatly enhances the performance of flux estimates even at half-hourly scale.

4 Conclusion

A new methodology was developed to estimate surface heat fluxes by assimilating land surface temperature (LST) and soil moisture observations using the particle batch smoother (PBS). The PBS uses all available observations within a window to update states and parameters in that window. The methodology was based on the surface energy balance and aimed to estimate two parameters: a bulk heat transfer coefficient (CHN) which scales the sum of surface heat fluxes, and evaporative fraction (EF) which represents the partitioning between sensible and latent heat fluxes. Two PBS strategies were implemented in this study. First, LST observations were assimilated into the force-restore model using the PBS to estimate surface heat fluxes. Second, to improve the estimation on wet or densely vegetated surfaces, soil moisture was modeled with a simple scheme to further constrain EF, and soil moisture observations were also assimilated.

The methodology was applied to data collected during the FIFE 87 and FIFE 88 experiments. When LST observations were assimilated, the estimated EFs successfully captured the daily variations in observed EFs, and the estimated EF values were much closer to the observations compared to OL results. The RMSEs of flux estimates were significantly reduced at both daily and half-hourly scales. The assimilation results were more accurate for drying events and less satisfactory for wetting events. When the surface was very wet or densely vegetated, the surface heat flux partitioning became more energy-limited, thus the coupling between EF and LST became weaker, and estimating EF from LST became very uncertain. In contrast, when the surface was dry, the coupling between EF and LST was tight, and LST was a strong constraint on surface heat flux partitioning.

When soil moisture observations were assimilated simultaneously, the EF estimates were greatly improved, particularly for wet days. When the surface was wet or densely vegetated, soil moisture constrained EF by forcing high EF values. The flux estimates were also improved at both daily and half-hourly scales, especially for H estimates. This implies that assimilating soil moisture observations greatly benefits parameter estimation. The time series of daily flux estimates demonstrated that the day-to-day variations of fluxes were more consistent with observations after assimilating soil moisture observations.

Results from a simulation test in terms of data availability showed that when only LST observations were assimilated, the RMSEs of flux estimates only improved marginally when over five LST observations were available each day. This is instructive in the context of using remote sensing data, in which the availability of observations is influenced by cloud cover. When soil moisture observations were assimilated, the flux estimates were significantly improved, particularly when LST observations were sparse.

This study demonstrated the potential value of joint assimilation of soil moisture and LST observations for flux estimation, and cast some light on potential applications with remote sensing data. However, the differences with a real remote sensing application should be noted. Our study used daily area-averaged soil moisture, in which the subdaily dynamics were smoothed, while satellites provide instantaneous soil moisture observations with a lower temporal resolution (2–3 days). In this study, a prior guess of CHN was available from previous studies. This information is generally not available, particularly over large areas. Future studies will incorporate the approach of Farhadi et al. [2014, 2016] to initialize CHN with LAI, which ensures that a prior guess of CHN can be obtained dynamically. The optimal β values may not be determined by trial and error due to the limited availability of flux observations. In that case, an adaptive PBS developed by Dong et al. [2016b] will be used, which determines the optimal β values automatically by maximizing the reliability of particle estimates. Issues such as spatiotemporal resolution, data accuracy, and spatial heterogeneity will also be addressed.

Acknowledgments

The first and second author were financially supported for their Ph.D. researches by the China Scholarship Council (CSC) with the project reference number 201306040112 and 201206040043, respectively. The FIFE data sets used in this study were acquired online from http://daac.ornl.gov. The authors thank the providers for the use of their data.

Ancillary