## 1. Introduction

[2] In parallel to the growing technologies for earth remote sensing, we have witnessed an increasing interest to improve the accuracy of observations and integrate them with predictive models for enhancing our environmental forecast skills. Remote sensing observations are typically noisy and coarse-scale representations of a true state variable of interest, lacking sufficient details for fine-scale environmental modeling. In addition, environmental predictions are not perfect as models often suffer either from inadequate characterization of the underlying physics or inaccurate initialization. Given these limitations, several classes of estimation problems present themselves as continuous challenges for the atmospheric, hydrologic, and oceanic science communities. These include (1) downscaling (DS), which refers to the class of problems for enhancing the resolution of a measured or modeled state of interest by producing a fine-scale representation of that state with reduced uncertainty; (2) data fusion (DF), to produce an improved estimate from a suite of noisy observations at different scales; and (3) data assimilation (DA), which deals with estimating initial conditions in a predictive model consistent with the available observations and the underlying model dynamics. In this paper, we revisit the problems of downscaling, data fusion, and data assimilation focusing on a common thread between them as variational ill-posed inverse problems. Proper regularization and solution methods are proposed to efficiently handle large-scale data sets while preserving key statistical and geometrical properties of the underlying field of interest, namely, non-Gaussian and structured variability in real or transformed domains. Here, we only examine a few hydrometeorological inverse problems with particular emphasis on land-surface applications.

[3] In land-surface hydrologic studies, DS of precipitation and soil moisture observations has received considerable attention, using a relatively wide range of methodologies. DS methods in hydrometeorology and climate studies generally fall into three main categories, namely, dynamic downscaling, statistical downscaling, and variational downscaling. Dynamic downscaling often uses a regional physically based model to reproduce fine-scale details of the state of interest consistent with the large-scale observations or outputs of a global circulation model [e.g., *Reichle et al*., 2001a; *Castro et al*., 2005; *Zupanski et al*., 2010]. Statistical downscaling methods encompass a large group of methods that typically use empirical multiscale statistical relationships, parameterized by observations or other environmental predictors, to reproduce realizations of fine-scale fields. Precipitation and soil moisture statistical downscaling has been mainly approached via spectral and (multi)fractal interpolation methods, capitalizing on the presence of a power law spectrum and a statistical self-similarity/self-affinity in precipitation and soil moisture fields [*Lovejoy and Mandelbrot*, 1985; *Lovejoy and Schertzer*, 1990; *Gupta and Waymire*, 1993; *Kumar and Foufoula-Georgiou*, 1993; *Perica and Foufoula-Georgiou*, 1996; *Veneziano et al*., 1996; *Wilby et al*., 1998a, 1998b; *Deidda*, 2000; *Kim and Barros*, 2002; *Rebora et al*., 2005; *Badas et al*., 2006; *Merlin et al*., 2006; among others]. In variational approaches, a direct cost function is defined whose optimal point is the desired fine-scale field which can be obtained via using an optimization method. Recently along this direction, *Ebtehaj et al*. [2012] cast the rainfall DS problem as an inverse problem using sparse regularization to address the intrinsic rainfall singularities and non-Gaussian statistics. This variational approach belongs to the class of methodologies presented and extended in this paper.

[4] The DF problem has also been a subject of continuous interest in the precipitation science community mainly due to the availability of rainfall measurements from multiple spaceborne (e.g., TRMM and GOES satellites) and ground-based sensors (e.g., the NEXRAD network and rain gauges). The accuracy and space-time coverage of remotely sensed rainfall are typically conjugate variables. In other words, more accurate observations are often available with lower space-time coverage and vice versa. For instance, low-orbit microwave sensors provide more accurate observations but with less space-time coverage compared to the high-orbit geo-stationary infrared (GOES-IR) sensors. Moreover, there are often multiple instruments on a single satellite (e.g., precipitation radar and microwave imager on TRMM), each of which measures rainfall with different footprints and resolutions. A wide range of methodologies, including weighted averaging, regression, filtering, and neural networks, has been applied to combine microwave and Geo-IR rainfall signals [e.g., *Adler et al*., 2003; *Huffman et al*., 1995; *Sorooshian et al*., 2000; *Huffman et al*., 2001; *Hong et al*., 2004; *Huffman et al*., 2007]. Furthermore, a few studies have addressed methodologies to optimally combine the products of the TRMM precipitation radar (PR) with the TRMM microwave imager (TMI) using Bayesian inversion and weighted least squares (WLS) approaches [e.g., *Masunaga and Kummerow*, 2005; *Kummerow et al*., 2010]. From another direction, Gaussian filtering methods on Markovian tree-like structures, the so-called scale recursive estimation (SRE), have been proposed to merge spaceborne and ground-based rainfall observations at multiple scales [e.g., *Gorenburg et al*., 2001; *Tustison et al*., 2003; *Bocchiola*, 2007; *Van de Vyver and Roulin*, 2009; *Wang et al*., 2011], see also *Kumar* [1999] for soil moisture applications. Recently, using the Gaussian-scale mixture probability model and an adaptive filtering approach, *Ebtehaj and Foufoula-Georgiou* [2011a] proposed a fusion methodology in the wavelet domain to merge TRMM-PR and ground-based NEXRAD measurements, aiming to preserve the non-Gaussian structure and local extremes of precipitation fields.

[5] Data assimilation has played an important role in improving the skill of environmental forecasts and has become by now a necessary step in operational predictive models [see *Daley*, 1993]. Data assimilation amounts to integrating the underlying knowledge from the *observations* into the first guess or the *background* state, typically provided by a physical model from the previous forecast step. The goal is then to obtain an improved estimate of the current state of the system with reduced uncertainty, the so-called *analysis*. The analysis is then used to forecast the state at the next time step and so on (see *Daley* [1993] and *Kalnay* [2003] for a comprehensive review). One of the most common approaches to the data assimilation problem relies on variational techniques [e.g., *Sasaki*, 1958; *Lorenc*, 1986; *Talagrand and Courtier*, 1987; *Courtier and Talagrand*, 1990; *Parrish and Derber*, 1992; *Zupanski*, 1993; *Courtier et al*., 1994; *Reichle et al*., 2001b; *Margulis and Entekhabi*, 2003; among many others]. In these methods, one explicitly defines a cost function, typically quadratic, whose unique minimizer is the analysis state. On the other hand, very recently, *Freitag et al*. [2012] proposed a regularized variational data assimilation scheme to improve assimilation results in advection-dominated flow in the presence of sharp weather fronts.

[6] The common thread in the DS, DF, and DA problems is that, in all of them, we seek an improved estimate of the true state given a suite of noisy and down-sampled observations and/or uncertain model-predicted states. Specifically, let us suppose that the unknown *true* state in continuous space is denoted by *x*(*t*) and its indirect observation (or model output), by *y*(*r*). Let us also assume that *x*(*t*) and *y*(*r*) are related via a linear integral equation, called the Fredholm integral equation of the first kind, as follows:

where is the known kernel relating *x*(*t*) and *y*(*r*). Recovery of *x*(*t*) knowing *y*(*r*) and is a classic linear inverse problem. Clearly, the deconvolution problem is a very special case with the kernel of the form , which in its discrete form plays a central role in this paper. Linear inverse problems are by nature ill-posed, in the sense that they do not satisfy at least one of the following three conditions: (1) existence, (2) uniqueness, and (3) stability of the solution. For instance, when due to the kernel architecture, the dimension of the observation is smaller than that of the true signal, infinite choices of *x*(*t*) may lead to the same *y*(*r*) and there is no unique solution for the problem. For the case when *y*(*r*) is noisy and has a larger dimension than the true state, the solution is typically very unstable because the high-frequency components in *y*(*r*) are typically amplified and spoil the solution in the inversion process. A common approach to make an inverse problem well posed is via the so-called *regularization* methods [e.g., *Hansen*, 2010]. The goal of regularization is to properly constrain the inverse problem aiming to obtain a unique and sufficiently stable solution. The choice of regularization typically depends on the continuity and degree of smoothness of the state variable of interest, often called the *regularity* condition. For instance, some state variables or environmental fluxes are very regular with high degree of smoothness and differentiability (e.g., pressure), while others might be more irregular and suffer from frequent and different sorts of discontinuities (e.g., rainfall). In fact, it can be shown that the proper choices of regularization not only yield unique and stable solutions but also reinforce the underlying regularity of the true state in the solution. It is important to note that different regularity conditions are theoretically consistent with different statistical signatures in the true state, a fact that may guide proper design of the regularization, as explored in this study.

[7] The central goal of this paper is to propose a unified framework for the class of DS, DF, and DA problems by recasting them as discrete linear inverse problems using a relevant regularization in the derivative space, aiming to solve them more accurately compared to the classic weighted least squares (WLS) formulations. From a statistical standpoint, the main motivation is to explicitly incorporate non-Gaussianity of the underlying state in the derivative domain as a prior knowledge to obtain an improved estimate of jump and isolated extreme variabilities in the time-space structure of the hydrometeorological state of interest. Note that the proposed framework relies on the seminal works by, for example, *Tibshirani* [1996], *Chen et al*. [2001], *Candes and Tao* [2006], and recent developments in mathematical formalisms of inverse problems [e.g., *Hansen*, 2010; *Elad*, 2010], which have received a great deal of attention in statistical regression and image processing, but are relatively new to the communities of hydrologic and atmospheric sciences. To the best of our knowledge, in these areas, the only studies that explore these methodologies are *Ebtehaj et al*. [2012] and *Freitag et al*. [2012] for rainfall downscaling and data assimilation of sharp fronts, respectively.

[8] The presented methodologies for the DS and DF problems are examined through downscaling and data fusion of remotely sensed rainfall observations, which have fundamental applications in flash flood predictions, especially in small watersheds [*Rebora et al*., 2005; *Siccardi et al*., 2005; *Rebora et al*., 2006]. We show that the presented methodologies allow us to improve the quality of rainfall estimation and reduce estimation uncertainty by recovering the small-scale high-intensity rainfall extreme features, which have been lost in the low-resolution sampling of the sensor. For the DA family of problems, the promise of the presented framework is demonstrated via an elementary example using the heat equation, which plays a key role in the study of land surface heat and mass fluxes [e.g., *Peter-Lidard et al*., 1997; *Liang et al*., 1999]. The results demonstrate that the accuracy of the analysis and forecast cycles in a DA problem can be markedly improved, compared to the classic variational methods, especially when the initial state exhibits different forms of discontinuities.

[9] Section 2 provides conceptual insight into the discrete inverse problems. Section 3 describes the DS problem in detail, as a primitive building block for the other studied problems. Important classes of regularization methods are explained and their statistical interpretation is briefly discussed from the Bayesian point of view. Examples on rainfall downscaling are presented in this section by taking into account the specific regularity and statistical distribution of the rainfall fields in the derivative space. Section 4 is devoted to the regularized DF class of problems with examples and results on remotely sensed rainfall data. The regularized DA problem is discussed in section 5. Concluding remarks and future research perspectives are presented in section 6. The important duality between regularization and its statistical interpretation is further presented in Appendix Statistical Interpretation, while Appendix Gradient Projection Method for the Huber Regularization is devoted to algorithmic details important for implementation of the proposed methodologies.