Time-lapse geophysical measurements are widely used to monitor the movement of water and solutes through the subsurface. Yet commonly used deterministic least squares inversions typically suffer from relatively poor mass recovery, spread overestimation, and limited ability to appropriately estimate nonlinear model uncertainty. We describe herein a novel inversion methodology designed to reconstruct the three-dimensional distribution of a tracer anomaly from geophysical data and provide consistent uncertainty estimates using Markov chain Monte Carlo simulation. Posterior sampling is made tractable by using a lower-dimensional model space related both to the Legendre moments of the plume and to predefined morphological constraints. Benchmark results using cross-hole ground-penetrating radar travel times measurements during two synthetic water tracer application experiments involving increasingly complex plume geometries show that the proposed method not only conserves mass but also provides better estimates of plume morphology and posterior model uncertainty than deterministic inversion results.
 Commonly used smoothness-constrained deterministic inversions used to image plume targets typically exhibit difficulty to accurately resolve plume mass and spread [e.g.,Singha and Gorelick, 2005; Day-Lewis et al., 2007; Doetsch et al., 2010]. Such inverse problems can be reformulated to conserve mass and to yield recovered plumes that are more compact [e.g., Ajo-Franklin et al., 2007; Farquharson, 2008], but they do not appropriately consider model nonlinearity during the inference of subsurface properties. Also, geophysical data are known to most frequently contain insufficient information to uniquely resolve spatially varying subsurface properties at a high spatial resolution, which means that multiple solutions that fit the data equally well are possible. Ideally, (hydro)geophysical inversion methods should consider this inherent nonuniqueness and provide an ensemble of model realizations that accurately span the range of possible (hydro)geophysical interpretations, with probabilistic properties that are consistent with the available data and prior information [e.g., Mosegaard and Tarantola, 1995; Ramirez et al., 2005].
 We present a novel stochastic inversion method to recover three-dimensional tracer distributions from time-lapse geophysical data, and provide consistent estimates of model uncertainty within a Bayesian framework. This approach employs a lower-dimensional model parameterization related to the Legendre moments of the plume [Teague, 1980; Day-Lewis et al., 2007]. During the inversion, the proposed moments are mapped into a tracer distribution (e.g., moisture content) which is subsequently transformed into a geophysical model (e.g., radar wave speed) using a petrophysical relationship. The geophysical model response (e.g., first-arrival travel times) is then simulated and compared with measurement data. To make sure that we accurately mimic expected plume mass and morphology, we do not work directly with the Legendre moments, but instead sample the null-space of the singular value decomposition (SVD) of a matrix containing predescribed mass and morphological constraints. This further reduces the dimensionality of the permissible model space thereby allowing for Markov chain Monte Carlo (MCMC) simulation to explore the posterior distribution of the plume geometry. We illustrate our method with two synthetic vadose zone water tracer application experiments involving increasingly complex plume shapes that are sampled using different amounts of cross-hole ground-penetrating radar (GPR) travel time data. We compare and evaluate our results against both classical least squares and compact mass conservative deterministic inversion results. To the best of our knowledge, this is the first mass conservative MCMC inversion of three-dimensional (3-D) tracer transport using time-lapse geophysical data.
 This paper is organized as follows. Section 2 presents the theoretical concepts underlying the proposed inversion approach. In section 3, we evaluate our method against state-of-the-art deterministic inversion techniques for a synthetic experiment involving a moisture plume with a relatively simple form similar toDoetsch et al. . This is followed in section 4 by an application to a much more heterogeneous moisture plume. Then, section 5 discusses important aspects that influence the performance of the method and suggests some further developments. Finally, section 6 draws conclusions about the presented work.
2. Inversion Methodology
 This section describes the inversion methodology developed herein. We first provide a detailed mathematical description of the Legendre moments and the singular value decomposition (SVD) approach used to describe, as closely and consistently as possible, the observed plume with the fewest possible number of model parameters. A brief overview of the MT-DREAM(ZS) algorithm and likelihood function used to sample the posterior parameter distribution is subsequently given. This section then concludes with a description of the deterministic geophysical inversion methods used to benchmark our results.
2.1. Background of Legendre Moments
 Geometric moments are commonly used to characterize geophysical or geophysically derived plumes [e.g., Singha and Gorelick, 2005; Day-Lewis et al., 2007]. The zero-order moment measures the total mass, the first-order moments describe the center of mass, the second-order moments are related to the spread, and higher-order moments characterize morphological features of non-Gaussian plumes. The geometric moments, , of a uniformly discretized three-dimensional image of size nx, ny, and nz, for which xi, yi, and zi are the spatial coordinates of the center of grid element i, are given by
where p, q, and r are the moment order in the x, y, and z directions, respectively, and , , and represent the voxel dimensions.
 Moment-based image reconstruction has extensively been investigated in the past several decades, and remains an active area of research today [Milanfar et al., 1996; Gustafsson et al., 2000; Pidlisecky et al., 2011]. Image reconstruction from orthogonal moments was first proposed by Teague for compact image description and reconstruction. Orthogonal moments have been found to provide more accurate reconstructed images than nonorthogonal geometric moments because higher-order geometric moments contain redundant information and show significant correlations [e.g.,Teague, 1980; Shu et al., 2006]. In the Teague  framework, orthogonal polynomials serve as basis functions for the orthogonal moments. Various possible orthogonal polynomials exist, for example: Hermite, Legendre, Jacobi, Laguerre, and Chebyshev polynomials [e.g., Chihara, 1978]. Of particular relevance to our work is the application of Legendre moments in the context of cross-hole geophysical inversion byDay-Lewis et al. . The Legendre moments, λ, of θ are given by
where , and signify the transformed model coordinates on a unit square grid: , , and represent the voxel dimensions of the unit square grid, and denotes the Legendre polynomial of order p evaluated by numerical integration over cell i in the x direction. We can rewrite equation (2) in matrix notation
where Pincludes the Legendre polynomial products on the three-dimensional unit grid. Note that coordinate transformation to the unit grid is required to preserve the orthogonality of . The geometric moments, , and are linearly related through the invertible matrix L and B such that: , and , where are geometric moments defined over the unit grid [Day-Lewis et al., 2007].
 We can reconstruct from its orthogonal Legendre moments at a given resolution defined by a truncated Taylor series expansion [Teague, 1980]
where the superscript stands for “reconstructed” and is the maximum order of moments used for the reconstruction. Equation (4) can be recast in matrix form
where contains the polynomial product coefficients of the orthogonal moments. By setting , then is , and element is given by the product: .
2.2. Tracer Characterization Using Constrained Legendre Moments
 We assume that the initial model is known and has a forward response d0 for a given experimental configuration. The change in geophysical response at some time t after application of the tracer, , is then used to infer the vector from , up to a predefined order in each spatial dimension. We can thus potentially reconstruct with a significantly reduced parameter dimensionality: from the original dimensionality of to a much lower dimensionality of npqr. However, many different models, , will likely exist that honor the experimental data, but do not necessarily conserve mass and may show aberrant shapes. To exclude such models, we introduce a transformed model space, which only contains models that conserve mass and honor prior constraints on plume morphology. Note that the constraints considered in this study are that the plume magnitude should be zero at the boundaries of the model domain, but could also include different levels of prior information, such as borehole logging data.
 Conservation of mass can be enforced by defining the first Legendre moment, , as
where (m3) signifies the total volume of the applied tracer.
 A system of equations can be defined that includes equation (6) and the restriction that along the six faces of the three-dimensional grid. Hence, if is the total number of the mass constraint and the prescribed constraints at the grid's corners and along edges and sides of the grid, then matrix A is and c is .
 To honor in the MCMC sampling, we use a singular value decomposition (SVD) of
where U is an orthogonal matrix with columns that are unit basis vectors spanning the space of constraints imposed in , V is a orthogonal matrix with columns that are basis vectors spanning the model space, and S is an diagonal matrix with singular values given as diagonal elements sorted in decreasing order. To avoid including numerical artifacts within the model space, we only retain those k singular values of the SVD of A that have a ratio larger than with respect to the largest singular value. For the examples considered herein, this ratio corresponds to the point at which singular values suddenly drop over 15 orders of magnitude to a plateau around . This leads to the following pseudoinverse solution, , of
where , , and are of dimension , , and , respectively. The general solution of is given by [e.g., Aster et al., 2005]
where the columns of ( ) span the model null-space and is a vector with the model null-space coefficients. By inferring the coefficients through MCMC sampling, we can reconstruct a distribution of that systematically honors the specified model constraints and the geophysical data. An added benefit of inferring α instead of λ is the further dimensionality reduction of the inverse problem from npqr to .
 To help resolving small plumes with simple shapes without resorting to higher-order moments, we add six shape parameters, , that define the subregion containing the plume. The parameters are the indices of the first and last subgrid elements in the x, y, and z directions: . Using these parameters makes it necessary to rescale the mass constraint over the new subgrid: , with, for example, . For numerical efficiency, we compute only once using the largest grid in which any possible anomalous region is expected to be contained. Before calculating the geophysical response, the proposed change in moisture variation, , is then linearly interpolated from the finer mesh of the anomalous subgrid ( , , ) to the coarser original mesh ( , , ).
2.3. Inference of Tracer Distribution Using MCMC Simulation of GPR Travel Times
 If we denote the vector of model parameters with , then each vector defines a three-dimensional soil moisture distribution, (m3 m−3), which can be converted into a geophysical model (e.g., a radar wave speed model, v (m s−1)), using a petrophysical relationship (e.g., the complex refractive index model, CRIM [Birchak et al., 1974]). The petrophysical parameters can either be fixed a priori or jointly estimated with . The GPR travel times d considered herein are calculated from through a finite difference algorithm [Podvin and Lecomte, 1991].
 We adopt a Bayesian viewpoint, and estimate the posterior distribution of using a distributed computing implementation of the MT-DREAM(ZS) algorithm [Laloy and Vrugt, 2012]. This MCMC algorithm uses DREAM [Vrugt et al., 2009] as its main building block, but implements multitry proposal and sampling from an archive of past states to further accelerate convergence to a limiting distribution. We use three different Markov chains with three to five parallel proposal points in each individual chain depending on processor availability. We assume a uniform prior distribution for all dimensions of and use the following standard Gaussian log likelihood function, , to summarize the fit between measured and geophysical model predicted travel times
where denotes the standard deviation of the measurement data error, and represent the initial and boundary conditions of the geophysical model. After convergence has been achieved, MT-DREAM(ZS) returns a large number of posterior samples from . In addition, models that violate physical constraints by containing soil moisture values larger than porosity, , or smaller than zero are assigned a very low log likelihood. Indeed, the constraints in preserve mass and prespecified morphological features, but do not prevent the generation of models containing values at one or more voxels that are negative or exceed porosity.
2.4. Classical and Mass Conservative Deterministic Inversions
 The performance of the MCMC inversion methodology presented in sections 2.1–2.3 is compared with results obtained by two different deterministic geophysical inversion approaches. In this particular case, the inverse problem is formulated to solve for the radar slowness (slowness is the inverse of velocity), s, finely discretized on a three-dimensional Cartesian grid. The standard deterministic least squares difference inversion uses an objective function (OF) of the form
where , with being the final residual vector obtained when inverting the background data set, [e.g., LaBrecque and Yang, 2001], is the forward model response, is the data covariance matrix (assumed to be described by an uncorrelated Gaussian function with constant variance, , is a trade-off parameter that determines the weight given to the model regularization term, is the reference slowness model obtained by inverting , and is the model covariance matrix. The inverse problem is herein solved at each iteration p + 1 by successive linearization around the previous model . The system of equations is solved in a least squares sense using the iterative conjugate gradient algorithm LSQR [Paige and Saunders, 1982]
with being the sensitivity (i.e., the Jacobian) matrix at . The regularization operator, , is based on an exponential covariance function and calculated following Linde et al. . The inversion proceeds by logarithmically decreasing at small steps from a very large value until a model is found that explains the data, as defined by a WRMSE of , with WRMSE defined as
with being the forward response of data element i for model . In the following, we refer to the inversion results using this methodology as DET.
 Least squares inversions of weakly nonlinear inverse problems have generally excellent convergence characteristics, but commonly used objective functions that penalize model structure with l2 norm measures tend to provide models with overly smooth contrasts [e.g., Ellis and Oldenburg, 1994]. One way to obtain more compact models, while still relying on gradient-based methods, is to add a reweighting matrix toequation (12), which forces the inversion to seek models that approximately minimize other norms, such as the l1 norm. To do so, we define a measure of model structure as . Using the perturbed Ekblom lp norm to mimic the l1 norm gives rise to the following entries in the diagonal reweighting matrix [e.g., Farquharson, 2008]
where signifies a small value with respect to regions in which is large and a suitable choice is . To make a fair comparison with the MCMC inversion results, we also include mass constraints and strongly penalize entries in that are negative (i.e., indicating unphysical decreases in moisture following the water application). The relation between s and moisture content, , can be expressed using (1) that describes how radar slowness is related to the effective relative permittivity, , and cl the speed of light in a vacuum, and (2) a suitable petrophysical model, such as the CRIM equation [Birchak et al., 1974]
where represent the relative permittivity of the sediment grains (s), water (w), and air (a). Combining and equation (15) for an update gives:
which expresses a linear relationship between and through a well-defined and constant scaling factor.Equation (16) implies that water mass constraints can be added to an update through a constraint in the inversion that only assumes that the functional form of the CRIM equation is correct.
 The iterative reweighting, mass constraints, and positivity constraints can be combined in a new least squares problem to minimize, at each iteration, similar to equation (12)
where the new trade-off parameters and are chosen by trial and error to assure that the constraints are fulfilled within a given tolerance (e.g., that the mass is conserved within four significant digits). The vector contains entries for all elements. The number of rows in corresponds to the number of negative entries in and each row contains zeros, except an entry of 1 corresponding to one of the negative entries. The stopping criterion is the same as for the classical deterministic difference inversion. In the following, we refer to the inversion results using this methodology as DET-ME.
 One way to assess the sensitivity of the final solution to data errors is to perform an additional iteration after finding the final model in which the residual vector, , is replaced with a random vector with the same distribution as the assumed error. By repeating this procedure for many realizations of the random error field it is possible to statistically determine the model error arising from the data errors [e.g., Alumbaugh and Newman, 2000].
3. Homogeneous Soil Plume
3.1. Dense Data Set
 Our first case study considers a synthetic, relatively homogeneous plume similar to Doetsch et al.  arising from water tracer injection in the vadose zone. We discretized the model domain of 6 m by 6 m (in the x-y plane) and a 10 m depth range into 360,000 cubic voxels with side length of 0.1 m. The porosity was assumed to be uniform at 0.32 and the initial moisture distribution at is at hydraulic equilibrium with increasing saturation from the surface to the bottom (see background trend in Figure 1a). Then, during a period of three days, 1.704 m3 of water was injected at x = 2 m, y = 2 m, and z= 3.5–4 m, forming a well-defined plume at the end of the injection (Figure 1a). We simulated synthetic GPR travel time data from four fully penetrating boreholes with spatial coordinates (1) m, m, (2) m, m, (3) m, m, and (4) m, m. Multiple offset gathers between the six possible borehole planes were calculated using 0.3 m intervals between antenna positions over the depth range 1.5–9 m below ground level for geometries with angles between the transmitting and receiving antennas within 45° from the horizontal. The resulting travel time data set comprised N= 5,103 observations, which were subsequently corrupted with a zero-mean Gaussian noise with standard deviation, , of 1 ns. Throughout the remainder of this paper, the wording “original” is used to refer to this N = 5,103 travel time data set. We assumed that the petrophysical model was known and we used the same parameters as in the CRIM equation (equation (15)), namely, , and .
 We applied our inversion for orders 2 to 5 in each spatial dimension, and allowed, for each order, a maximum computational time equivalent to 30,000 times the time needed to build a proposal model and perform the subsequent forward solver call (approximately 40 s, a so-called computational time unit, or CTU). Characteristics of these MCMC runs are summarized inTable 1, which lists in separate columns the problem dimensionality, prior parameter range, acceptance rate, and convergence criteria of Raftery and Lewis  and Gelman and Rubin : RL and , respectively (see Laloy and Vrugt for details). The lowest-order model converges rapidly and exhibits a relatively high acceptance rate of proposal points in the different Markov chains. Higher orders (i.e., more model parameters) improve the data fit, but at the expense of a larger computational time needed to derive the posterior distribution of .
Table 1. Main Characteristics of the Markov Chain Monte Carlo (MCMC) Sampling Runs of the Homogeneous Soil Plume Case Study for Orders 2, 3, 4, and 5 Using 5,103 Data Pointsa
and are the number of inferred and parameters, respectively, denotes the prior range of the parameters, RLCTU and are the number of computational time units (CTU) necessary for each parameter to converge toward the posterior target according to the Raftery and Lewis burn-in andGelman and Rubin  convergence criteria, respectively, and AR is the mean acceptance rate. Bounds of the uniform prior probability density function (pdf) of β are set such that the anomalous subregion can occupy any subregion around the boreholes. N/A, not applicable.
To speed up convergence, the order 5 sampling run was initialized with the most likely model of order 4.
Figures 1b–1d and 2b–2d present slices of the most likely models obtained for orders 2, 4, and 5. Table 2 lists the corresponding image statistics, such as mass conservation error, error in center of mass, spread, and the RMSEθ that quantifies the difference between the true moisture plume and the most likely models. We find that the tracer mass is conserved for all orders within a negligible error level. This is not the case for classical deterministic inversion models that may show mass balance violations above 50% [e.g., Binley et al., 2002; Day-Lewis et al., 2007; Doetsch et al., 2010]. The center of mass is very well approximated for all orders with an error ranging from 0.05 to 0.07 m. Although none of the considered models perfectly reproduce the original image, order 4 and 5 both represent the plume morphology quite well with a RMSEθ of 0.005 (m3 m−3) and WRMSE of about 1.01 and 1.00, respectively.
Table 2. First- and Second-Order Statistics of the True Homogeneous Soil Moisture Plume, the Most Likely Plumes Obtained With Orders 2, 3, 4, and 5, and the Ones Obtained With the Classical Least Squares (DET) and Compact, Mass-Constrained, and Nonnegative (DET-ME) Deterministic Inversions Using 5,103 Data Pointsa
Error in Mass (%)
Center of Mass (m)
Error in Center of Mass (m)
Standard Deviation (m)
RMSEθ (m3 m−3)
Statistics derived from the classical least squares deterministic inversion (DET) rely on a somewhat arbitrarily defined cutoff value of +1.6%, that is, 1/10 of the generated maximum value. As mass conservation is enforced through a penalty term within the objective function for the nonsmooth deterministic inversion (DET-ME), no cutoff is used for the latter. WRMSE is the weighted root-mean-square error associated with the data misfit. RMSEθ quantifies the difference between the true and most likely moisture plumes for each stochastic and deterministic inversion. N/A, not applicable.
 To benchmark our results, we performed both a classical least squares deterministic inversion (see equations (11) and (12)) and a compact mass conservative deterministic inversion (see equations (14)–(17)). These optimization methods are sequential and one forward run thus consumes one CTU, and fitting the data to the error level typically requires 5–10 CTU. The resulting moisture models are shown in Figures 1e, 1f, 2e, and 2f with corresponding plume statistics summarized in Table 2. The least squares deterministic plume (DET) underestimates the actual mass with 6%, exhibits a 0.17 m error in center of mass, and overestimates plume spread with 73%. While the compact mass-enforced inversion (DET-ME) perfectly conserves mass, it presents an increased error in the center of mass of about 0.32 m, overestimates spread with 53% and shows about a twice as high RMSEθcompared to the most likely models of orders 4 and 5. Such degraded 3-D morphological approximations from deterministic inversions are not surprising as one can mainly expect to resolve information on plume shape along the GPR borehole planes only, whereas the deterministic models in the surroundings region are strongly affected by the regularization used. Hence, atx = 2 m (Figure 1), one of the diagonal borehole planes crosses the center of the plume and its shape is therefore rather well constrained by the data, while the measured data carry no information about zones where none of the 6 borehole planes hits the plume. This is illustrated in Figure 2, where the deterministic two-dimensional vertical representations of the plume atx= 3 m are quite poor, especially for the mass-constrained inversion (DET-ME) which spreads mass all around. In contrast, by including the size of the anomalous subregion ( vector) in the optimization problem and using fewer model parameters, our stochastic inversion approach enforces significant mass concentration in areas with no ray coverage.
Figure 3 presents histograms of the marginal posterior moisture distribution corresponding to orders 2 to 5 at two different locations in the spatial domain. The posterior soil moisture uncertainty becomes larger with increasing parameter dimensionality: from nearly 0% (in moisture content) for order 2 to about 7% for order 5 at x = 3 m, y = 2 m and z= 4 m. This provides a nice illustration of the well-known trade-off between model resolution and variance in inverse problems [e.g.,Jackson, 1972]. The solution for order 2 is very well resolved and the corresponding acceptance rate is relatively high: almost 28%. For this order, we sample only the six shape parameters of as the model null-space contained in is empty. For higher orders, we find that the moisture uncertainty is overall larger at x = 3 m, y = 2 m and z = 4 m (Figures 3e–3h) than at x = 2 m, y = 2 m and z = 4 m (Figures 3a–3d), highlighting that posterior uncertainty increases at locations with no ray coverage. At the former location, orders 4 and 5 denote uncertainties of 6% and 7% in moisture, respectively, compared with 3% and 4% for the latter location.
 The order 4 and 5 histograms encapsulate the true soil moisture values (indicated with crosses), though at the margin of the distributions. This deviation is the effect of model structural deficiencies introduced by an insufficient number of parameters to perfectly characterize the observed plume. Indeed, we can only expect to recover a decomposed version of the true soil moisture model at the considered order (see equation (4)). Hence, the higher the order, the better the potential approximation, but also the more difficult the MCMC sampling problem and the larger the resulting model uncertainty. Using the most likely parameters, one can compute the truncated true plume for each order considered. Doing so, it is observed (not shown) that the order 4 and 5 truncated true images look visually very similar to the true plume. Yet, the truncated true plumes of order 2 to 5 systematically violate physical constraints with a small percentage of voxels whose moisture values exceed porosity, thus leading to the automatic elimination of such models in the MCMC search (see section 2.3).
 It is worth emphasizing that the moisture content values inferred from the deterministic inversions at the two locations investigated in Figure 3 are too far removed from their true values to be conveniently plotted in Figure 3: 0.268 m3 m−3 (x = 2 m, y = 2 m and z = 4 m) and 0.213 m3 m−3 (x = 3 m, y = 2 m and z = 4 m) for DET, and 0.341 m3 m−3 (x = 2 m, y = 2 m and z = 4 m) and 0.199 m3 m−3 (x = 3 m, y = 2 m and z= 4 m) for DET-ME. With respect to the deterministic uncertainty characterization described insection 2.4, both deterministic methods provide an estimated model error in the range of 0.5%–0.7% in moisture at the 2 locations. More generally, the estimated uncertainties from the deterministic inversions typically do not exceed 1.0%. These values are indeed very small given the high number of degrees of freedom associated with the deterministic inversions: 45,000 (for numerical reasons, the full 360,000-dimensional domain was reduced by a factor two in each spatial direction before initiating the deterministic inversions). In comparison, our stochastic inversion technique reveals 3 to 4 times larger moisture uncertainties for a 32-dimensional search space only (order 4). The explanation for this apparent paradox is the tremendous influence of the model regularization term in the deterministic inversion that reduces the effective number of degrees of freedom quite substantially. This also implies that the estimated moisture values from the deterministic inversions will change dramatically if another regularization is used. Moreover, note that the interpretation of these uncertainties is fundamentally different as they quantify the impact of the data errors in estimating the most regularized model that fits the data.
3.2. Sparse Data Set
 The 5103 measurement data points that were used in section 3.1 takes several hours to collect in the field. A long data acquisition time might induce artifacts in the inversion if changes in the plume occur during this time period [e.g., Day-Lewis et al., 2003]. To further investigate this issue, we down-sampled the original data set, and repeated the numerical inversions using sources and receivers placed at every 0.9 m instead of 0.3 m along the boreholes. The resulting data set constitutesN = 567 travel time observations, roughly ten times less than the original data set used hitherto.
 Similarly as for the dense data set, Table 3 lists characteristics of the different MCMC trials, while Figures 4 and 5 present different slices of the most likely stochastic and deterministic models, and Table 4quantifies the corresponding image statistics. For this example, the compact mass conservative deterministic inversion (DET-ME) does a much better job in mimicking the plume shape and properties than the classical least squares inversion (DET), and we therefore restrict our attention to the results of DET-ME.
Table 3. Main Characteristics of the MCMC Sampling Runs of the Homogeneous Soil Plume Case Study With 567 Data Points for Orders 2, 3, 4 and 5 Onlya
RLCTU ( 103)
, , , , , RLCTU, , and AR are as in Table 1. N/A, not applicable; NC, not converged within the allowed 30,000 CTU.
To speed up convergence, the order 5 sampling run was initialized with the order 4 most likely model.
Table 4. First- and Second-Order Statistics of the True Homogeneous Soil Moisture Plume, the Most Likely Plumes Obtained With Orders 2, 3, 4, and 5, and the Ones Obtained With the Compact, Mass-Constrained, and Nonnegative (DET-ME) Deterministic Inversions Using 567 Data Pointsa
 Also for this case, the stochastic models better approximate the true image than the deterministic models. As expected, the higher-order models are not as well constrained by the down-sampled data set as for the original data set. This is illustrated inFigure 6, which depicts histograms of the posterior moisture uncertainty for the same two voxels as in Figure 3. Indeed, the travel times lead to a much larger model uncertainty, up to 11.5% in moisture content for orders 4 and 5 (Figures 6g and 6h), which translates into posterior models than can take quite different shapes, while still consistently honoring the data (see Figure 7). This is especially true for the order 5 posterior plumes that may be further removed from the true tracer morphology than the much simpler order 2 inferred plumes (see the associated RMSEθ values of Table 4). Particularly for the lower-order models, the use of a down-sampled data set increases the convergence speed of MCMC simulation with MT-DREAM(ZS). In practice, formal convergence is often difficult to assess. This is evident from the reported Raftery and Lewis  and Gelman and Rubin  statistics (Table 3).
 Compared to the marginal posterior probability density functions (pdf's) depicted in Figure 6, the estimated moisture values from the deterministic inversions are again too far removed from their synthetic true values to be conveniently plotted: 0.324 and 0.175 m3 m−3 at x = 2 m, y = 2 m, and z = 4 m and x = 3 m, y = 2 m, and z= 4 m, respectively. Furthermore, the deterministic errors remain always lower than 1%. In contrast, the arguably statistically more sound Bayesian inversion methodology predicts 6 to 11 times larger model parameter uncertainties for the 32-dimensional and 69-dimensional search spaces of orders 4 and 5, respectively.
4. Heterogeneous Soil Plume
 Our second case study focuses on a more heterogeneous synthetic plume, which was obtained by simulating water tracer infiltration in a layered unsaturated soil with the three-dimensional flow simulator HYDRUS 3D [Šimůnek et al., 2011]. At the beginning of the simulation, 1.365 m3 of water was applied to a very small surface area at x = 3 m, y = 3 m, and z = 0 m during the first day. This resulted in a rather complex plume shape two days later (Figure 8a). The same GPR setup, initial moisture distribution and data corruption were used as for the homogeneous soil plume in section 3.1, but the value of in the petrophysical model was changed to 0.41.
 Given the large morphological complexity of the plume, we only consider order 4 and 5 using a maximum computational time budget of 30,000 CTU for each individual MCMC trial. In addition, both classical least squares (DET) and compact, mass-constrained (DET-ME) inversions were performed.Table 5 summarizes the characteristics of the different MCMC trials, whereas Table 6 lists the most likely (MCMC inversion) and optimal (deterministic inversion) model statistics. None of the stochastic or deterministic models mimic the true plume as well as for the homogeneous soil case study. Figures 8b, 8c, 9b, and 9cpresent two-dimensional slices of the “best” deterministic models, whereasFigures 8d–8f, 9d–9f, 10b–10d, and 11b–11d show various realizations of the order 4 and 5 posterior distributions at similar locations. As expected, the deterministic models are overly smooth, whereas the stochastic models overall describe a considerable soil moisture variability.
Table 5. Main Characteristics of the MCMC Sampling Runs of the Heterogeneous Soil Plume Case Study for Orders 4 and 5 Using 5103 Data Pointsa
Table 6. First- and Second-Order Statistics of the True Heterogeneous Soil Moisture Plume, the Most Likely Plumes Obtained With Orders 4 and 5, and the Ones Obtained With the Classical Least Squares (DET) and Compact, Mass-Constrained, and Nonnegative (DET-ME) Deterministic Inversionsa
Error in Mass (%)
Center of Mass (m)
Error in Center of Mass (m)
Standard Deviation (m)
RMSEθ (m3 m−3)
Statistics derived from the classical least squares deterministic inversion (DET) rely on a somewhat arbitrarily defined cutoff value of +1.2%, that is, 1/10 of the generated maximum value. RMSEθ is defined in Table 2. N/A, not applicable.
 The two orders considered herein perform better than the deterministic inversions with respect to plume morphology. The center of mass estimates are indeed about three times better approximated by the different stochastic models compared to the DET model, and approximately two times better estimated compared to the DET-ME model (Table 6). The improvements are even much larger in terms of plume spread: the error standard deviations are 0.31 and 0.09 m for orders 4 and 5, respectively, compared with 1.80 m for DET and 1.23 m for DET-ME (Table 6). Among the most likely stochastic models, order 5 provides the lowest RMSEθ (0.0085 m3 m−3), indicating that this order provides the most satisfying results judging from this criterion.
 Another important observation is that, although the three-dimensional posterior moisture distribution is relatively similar for each considered order (Figures 8–11), the associated uncertainties are quite different and range from 6% to 8% in moisture for the orders 4 and 5, respectively (not shown). This is considerably higher than the uncertainties derived with the deterministic inversions, which do not exceed 1%.
 The corresponding truncated models of the true plumes were computed using a manually defined anomalous subregion. This was necessary as the most likely parameter set does not exactly contain the true heterogeneous soil plume. These truncated representations of the true model appear visually similar to the real true plume (not shown). Contrary to the homogeneous soil plume, there are no voxels with moisture value exceeding porosity because the infiltrated amount of water is smaller and the selected medium porosity is larger (0.41). Also, the order 4 and 5 truncations of the true plume exhibit insufficient probability to be part of the posterior distribution. The order 4 and 5 truncated true models lead to RMSE values of 1.066 and 1.021 ns respectively, which are substantially removed from their counterparts estimated with the stochastic inversion that range between 1.012 and 1.014 ns (order 4) and 1.008 and 1.010 ns (order 5). These differences may seem small, but with N = 5103 data points and a measurement error ( ) of 1 ns, a direct jump from 1.010 ns (worst solution for order 5 stochastic model) to 1.021 ns (order 5 truncated true model) has a probability of less than 1 10−23, which is practically zero.
 We have developed a dimensionality-reduced stochastic inversion technique that explores the posterior distribution of a three-dimensional tracer plume given geophysical data. The method conserves mass and prior morphological constraints can be included. This is achieved by using a state-of-the-art MCMC scheme to sample the null-space of a matrix of constraints defined in terms of the Legendre moments of the tracer distribution. To increase the effectiveness of the model reduction for low-order moments only, we include the dimensions of the subregion containing the plume in the sampling along with other variables. To the best of our knowledge, this is the first mass conservative MCMC inversion of three-dimensional tracer distribution using time-lapse geophysical data. Our results demonstrate that our method works well for a synthetic plume in a uniform media, and provides encouraging results for a more complex heterogeneous soil plume. One assumption underlying this work is that the background model is perfectly known. This assumption can be relaxed when working with real data without causing any significant deteriorations of the inferred changes with respect to background conditions. For real applications, one would use the data vector defined in section 2.4 instead of .
 The uncertainty estimates obtained by the proposed approach represent, for a given order and hence resolution, essentially all possible models that are consistent with the data and imposed constraints. The inherent spatial regularization implies that for most relevant applications, the model space will not contain the exact true model. This is an inherent feature of using a reduced model parameterization that needs to be considered when evaluating posterior model uncertainties.
 Synthetic experiments makes it possible to derive the truncated true plume for each considered order of the Legendre moments by using the most likely or manually determined shape parameters characterizing the anomalous subregion. Comparing the true truncated model with the derived posterior pdf could then potentially provide useful information about the effectiveness of the inversion. However, for the orders 2 to 5 considered herein, the log likelihood of the decomposed true model is insufficient to be part of the posterior distribution within our Bayesian framework. This happens because of two main effects: (1) the presence of a small proportion of voxels within the truncated true models with moisture content exceeding porosity, which causes them to be ignored in the MCMC search, and (2) the truncated models of the true model may result in forward simulations that do not fit the measurements as well as other candidate points in the permissible model space. In this context, it is important to realize that apparently very small changes in RMSE values represent very large differences in the associated log likelihood function value. In general, the smaller the assumed measurement error and the higher the number of data points, the peakier the log likelihood function becomes. For example with the Gaussian log likelihood function used herein with a measurement error, ns, and N = 5,103, a value of 1.02 ns is well removed from the posterior RMSE range. This implies that it is very important to accurately characterize the data errors. Ideally an additional error term should be added that quantifies the upscaling error caused by the truncation at a given order. This topic is outside the scope of the present work, but needs to be carefully addressed in future research.
 An important question for real-world applications concerns the order to be considered. For real systems, one has often very limited information about the expected plume shape and measurement errors are typically only known approximately. Increasing the order and thus the degrees of freedom (parameters) of the model will help improve the data fit, but at the expense of an increasing risk of generating complex models that overfit the data and might not well approximate the true plume. Using model selection criteria, such as the Bayesian information criterion (BIC) [Schwartz, 1978] or the Akaike information criterion (AIC) [Akaike, 1974], might help resolve this trade-off. As those criteria may not always be valid for nonlinear forward models, marginal likelihood estimation [Gelman and Meng, 1998; Mackay, 2003] seems however more attractive, though computationally challenging. Another interesting tool for parsimonious model identification is the reversible jump or transdimensional MCMC framework [Green, 1995; Sambridge et al., 2006] in which the number of degrees of freedom of the model is considered as uncertain as well. Both approaches will be explored in future studies.
 For the examples considered herein, our results suggest that order 4 is a good compromise choice for relatively simple plume shapes. For a highly heterogeneous tracer distribution, order 5 appears to be preferable over order 4. Even if our parameterization allows for a tremendous reduction in parameter dimensionality of the inverse problem (3 to 4 orders of magnitude) compared with the deterministic inversions, computational time remains an issue. Using a distributed computer network, it still typically requires from 5 to 12 days of calculations for the MCMC sampling to appropriately converge.
 Another crucial point concerns the prior distribution of the model null-space coefficients (α). In this work, we assumed a uniform initial distribution between −0.1 and 0.1. This range proved to be sufficiently large and encapsulated posterior distribution.
 Lastly, please note that the results of the heterogeneous soil plume would have further improved if we would have used stronger constraints on plume morphology. Yet, we purposely opted to present a general Bayesian analysis and sampling framework that requires little prior information. This simplifies application of the method herein to other (hydro)geophysical inverse problems.
 We have introduced a mass conservative MCMC inversion to derive statistical information about three-dimensional tracer distributions using time-lapse geophysical data. Two synthetic vadose zone water tracer experiments involving increasingly complex plume morphologies demonstrate that the proposed method conserves mass and yields improved representations of plume morphology compared with state-of-the-art deterministic inversions. The posterior soil moisture uncertainties are not only found to be more realistic and much larger than classical deterministically derived uncertainty estimates, but are also increasing with increasing order of the Legendre moments used to defined the permissible model space. Notice that the proposed approach has widespread potential to be used for the detection of and characterization of anomalies, and is thus not limited to hydrogeophysics. Our future work will apply the methodology presented herein to real-world geophysical data from field experiments.
 We are grateful to Andrew Binley for providing us with the moisture content model used in the homogeneous soil plume example. The Associate Editor Andrew Binley and three anonymous reviewers are thanked for their most constructive comments. Computer support, provided by the SARA center for parallel computing at the University of Amsterdam, Netherlands, is highly appreciated.