Multi-scale correlation functions associated with polynomials of the diffusion operator


  • Max Yaremchuk,

    Corresponding author
    1. Naval Research Laboratory, Stennis Space Center, Mississippi, USA
    • Naval Research Laboratory, Bldg 1009, Stennis Space Center, MS 39529, USA.
    Search for more papers by this author
    • The contributions of these authors to this article were prepared as part of their official duties as United States Federal Government employees.

  • Alexei Sentchev

    1. Laboratoire d’Oceanologie et de Geosciences, Wimereux, France
    Search for more papers by this author


Correlation functions associated with the inverse covariances represented by polynomials of the homogeneous diffusion operator D are obtained analytically for an arbitrary polynomial of D, constrained by the positive-definiteness condition of the covariance operator. Copyright © 2012 Royal Meteorological Society

1. Introduction

Oceanic and atmospheric processes are characterized by motions of different scales which often manifest themselves by distinct maxima in the spectral density of a background model state and in the spectra of background error covariance (BEC) matrices. Numerical modelling of the multi-scale BEC structures in variational data assimilation is a challenging task which has recently drawn a significant attention due continuous growth of computer power. In particular, it is desirable to formulate the scale-dependent BEC operators (Dance, 2004), which can account for smaller-scale components present in very high-resolution models. The term ‘multi-scale correlation function’ is also used in the theory of turbulence and reflects covariances of multifractal nature characterized by the power-law decay of correlations (e.g. Mandelbrot, 1997).

A straightforward way to construct multi-scale BEC operators is to use suitable superpositions of the single-scale correlation functions for modelling the BEC matrix elements (e.g. Hristopulos, 2003; Gaspari et al., 2006). In this approach, the resulting spectrum is difficult to control directly by the free parameters of the correlation functions and care should be taken to maintain positive definiteness of the correlation matrix.

A promising approach is to introduce scale separation in the BEC models by splitting the covariance matrix into several additive single-scale components (e.g. Wu et al., 2002; Purser et al., 2003) and perform assimilation on a sequence of grids with increasingly fine resolution (Li et al., pers. comm. 2012).

A multi-scale BEC operator can also be constructed using a polynomial of the discretized diffusion operator for representing the inverse covariance. This approach has been studied by many authors (e.g. Sasaki, 1970; Wahba and Wendelberger, 1980; Purser, 1986; McIntosh, 1990; Xu, 2005). Its attractive features are the flexibility in controlling the BEC spectrum and the low cost of computing the action of the inverse BEC matrix on a state vector. In practice, however, applications of this approach were limited to BEC operators with Gaussian-shaped correlation functions and their approximations (e.g. Weaver et al., 2003; Di Lorenzo et al., 2007). Among the reasons for that limited applicability is poor conditioning of the BEC operators generated by high-degree polynomials and the necessity to link polynomial coefficients with the shape of the BEC spectrum. In the recent studies of Hristopulos and Elogne (2007, 2009) and Yaremchuk and Smith (2011), correlation functions associated with an arbitrary quadratic polynomial of the homogeneous diffusion operator were obtained and relationships between the polynomial coefficients and the magnitude/length scale of the corresponding spectral peak have been provided.

In this note the result of Yaremchuk and Smith (2011) is extended for the case of an arbitrary polynomial, generating a multiple-peak BEC spectrum. Besides, it is shown that the action of the BEC operator can be reduced to a sequence of inversions of the quadratic functions of the diffusion operator, thereby relaxing the above-mentioned conditioning problem.

2. Homogeneous multi-scale correlation functions

This note deals with covariance modelling in equation image, although the results can be extended to an arbitrary differentiable manifold of constant curvature.

A general form of the inverse BEC operator built as a polynomial of the homogeneous diffusion operator Δ is

equation image(1)

Here I is the identity operator and αi are real numbers, constrained by the positive definiteness requirement of B−1, which can be taken into account explicitly by diagonalizing B−1 via the Fourier transform. In the Fourier representation the inverse BEC operator acts as multiplication by the polynomial in k2 ≡ |k|2 (k is the wavenumber), and the positive-definiteness property translates into the requirement for the spectral polynomial

equation image(2)

to be positive for all k2 > 0. This constraint is equivalent to the statement that the right-hand side of Eq. (2) must not have real positive roots. A particular form of the even-order polynomial satisfying this requirement is

equation image(3)

where M = L/2,

equation image(4)

where overline in Eq. (3) denotes complex conjugate and zm = am + ibm are arbitrary complex numbers with ambm0. In its general form, the polynomial (3) is additionally multiplied by the product over the arbitrary number of real negative roots. To simplify the formulas, we consider this case in the Appendix, and focus on the analysis of Eq. (3) omitting the product (summation) limits over m and assuming there are no real negative and multiple roots. The latter requirement is not restrictive for practical purposes, because location of the roots is never known exactly, and the BEC spectrum can be well approximated by Eq. (3) (see Appendix).

It is instructive to note that expression (3) can also be rewritten in the form

equation image(5)

Compared to the spectral representation (Eq. (2)), representation (5) has the advantage that its free parameters are not constrained by the positive-definiteness requirement and can be interpreted as the scales (equation image) and magnitudes (equation image) of the modes forming the spectrum.

Since the reciprocal of B−1(k) provides the spectral representation of the BEC operator, the matrix elements B(r) of B (covariance functions) depend only on the distance r from the diagonal and can be obtained in the form of a single integral over k:

equation image(6)

Here J denotes the Bessel function of the first kind, n is the dimension of the physical space and s = n/2 − 1. Equation (6) is obtained by substitution of the reciprocal of Eq. (3) into the integral of the inverse Fourier transform and integrating over the solid angle in the wavenumber space (e.g. Yaremchuk and Smith, 2011). The integral (Eq. (6)) can be taken by decomposing

equation image(7)

into elementary fractions:

equation image(8)


equation image(9)

By replacing Eq. (7) in Eq. (6) with the sum (8) the integral is reduced to the sum of Hankel–Nicholson type integrals (e.g. Abramowitz and Stegun, 1972, eq. 11.4.44) and can be taken explicitly. The cited integral is valid if the following constraints are satisfied: (a) −1 < s < 3/2 and (b) the real part of zm is positive. The first condition is met in the practical cases of 0 < n < 5. Furthermore, in view of Eq. (5) both am and bm can be assumed to be positive without loss of generality and thus the second constraint is also satisfied. The resulting expression for B(r) is

equation image(10)

where ρm = zmr, K stands for the modified Bessel function of the second kind, and angle brackets denote taking the real part.

The corresponding correlation functions Cn(r) are obtained through normalizing Eq. (10) by Bn(0). For n < 4 the BEC function values at r = 0 are

equation image(11)
equation image(12)
equation image(13)

In one- and three-dimensional cases the correlation functions can also be expressed in terms of the exponents:

equation image(14)
equation image(15)

Relationships (10)–(13) provide analytical expressions for the multi-scale homogeneous BEC functions and the corresponding CFs. In many applications, it is often important to know the value N of the convolution of B with the δ-function at r = 0 (the normalization factor) which is used for constructing the BEC operator numerically. The factor can be found by integrating Cn(r) over equation image:

equation image(16)

3. Practical issues

In applications, a BEC model is often constructed by fitting spectral (7) or correlation (10) functions to those derived from experimental data. The random fields under consideration are characterized by 2m parameters, which give enough freedom for approximating complex spectra. The approximation procedure can be formulated as a least squares problem in 2m dimensions, which may be rather complicated due to nonlinearity of B with respect to the fitting parameters am and bm. Therefore it is useful to have guidance on how the magnitudes and locations of the model peaks are related to the scales and amplitudes of the physical modes contributing to the experimental spectrum (Figure 1).

Figure 1.

An example of the normalized spectrum B(k) (Eq. (7), left panel) and the respective correlation function B(r)/B(0) (Eqs (10, 12), right panel) for M = n = 2, z1 = .5 + 3i; z2 = .2 + 6i.

The contribution of the mth mode to the spectrum can be assessed by integrating the right-hand side of Eq. (8):

equation image(17)

In the limit when distances |blbm| between the spectral peaks of B are much larger than their half-widths am (i.e. am/bm Lt; 0 in particular), Eq. (17) can be simplified using the asymptotic approximations

equation image

so that

equation image(18)

Asymptotic values of spectral density at the peaks are respectively

equation image(19)

i.e. the peak amplitudes are inversely proportional to equation image and to the square of the mode scale equation image. Expressions (17)–(19) can be useful in generating the first-guess values for zm to initialize an iterative procedure of approximating experimental data.

After the model parameters are established, the action of the inverse BEC operator can be computed recursively by

equation image(20)

The inverse BEC model (Eq. (20)) can then be employed either to compute the action of B using an iterative inversion scheme or to directly compute the gradient of a 3dVar cost function involving the quadratic form xTB−1x, where x is the state vector.

The considered multi-scale BEC operators can be used in many oceanographic applications, where the background errors have multi-scale spectra. For instance, surface waves are often characterized by two-peak spectra generated by the swell and locally forced wind waves. Filtering such a wave-induced signal from observations is important in many applications (e.g. vertical positioning of the autonomous underwater vehicles, turbulence microstructure measurements in shallow seas).

Figure 2(a) demonstrates typical velocity spectra, derived from observations by an upward-looking bottom-mounted acoustic Doppler current profiler (Korotenko et al., 2012). Measurements were taken in the period of well-developed wind waves with a dominant frequency f ∼ 0.2 Hz superimposed on the swell (f ∼ 0.1 Hz) propagating from the Bay of Biscay. Slight asymmetry of the beam directions with respect to the vertical prevents cancellation of the wave-induced orbital motions in averaging over the beams and contaminates the turbulence spectrum with a double-bump feature seen in Figure 2(a).

Figure 2.

(a) Power spectrum of the cross-shore velocities near Boulogne-sur-Mer on 15 June 2009 (black line) measured at 1 s resolution. Contamination by surface waves is seen as a pronounced double peak. Grey line shows the spectrum of filtered series. (b) Solid black line is the ratio between the observed spectrum and its power-law interpolation (dashed line in left panel) within the wave-contaminated band ω. This ratio was used for constructing the filter F, whose spectrum is shown in grey.

Impact of the surface waves can be removed by constructing a rational filter (e.g. Antoniou, 2000) equation image using the polynomials (1) of the diffusion operator Δ = tt. The rational function F(k) can be obtained by adjusting the filter parameters equation image to the ratio between the observed spectrum and its power-law approximation (dashed line in Figure 2(a)) in the wave-contaminated frequency band ω = [0.06 − 0.4] Hz. After the adjustment, the matrix elements of B are computed using Eqs (10), (14):

equation image(21)

whereas the action of equation image is given by Eq. (20):

equation image(22)

The filter is implemented by differentiating the series with Eq. (22) and then smoothing it with the kernel (Eq. (21)).

Figure 2(b) demonstrates the result of fitting the filter parameters equation image to the wave-induced part of the spectrum shown in Figure 2(a). The fit has a relative error of 7% within the target band of 0.06–0.4 Hz. In the same frequency band, the filtered series spectrum has similar (6%) deviation from the power law when normalized by deviation of the observed spectrum from that law. Proximity of the filtered spectrum to the power law can be further improved by increasing the order M of the polynomials B−1(k) and equation image.

4. Summary and discussion

Analytical expressions for the matrix elements of homogeneous BEC operators generated by the polynomials (Eq. (1)) of the diffusion operator are obtained. The considered BEC operators can be used in geophysical applications involving multi-scale phenomena whose contribution to the spectrum can be modelled by adjusting the free parameters (polynomial coefficients) of the BEC model. Applicability of the technique to a simple two-scale one-dimensional filtering problem has been demonstrated.

A particular advantage of the considered type of BEC operators is the fact that their inverses can be represented by sparse matrices that can be efficiently implemented on the grids of various complexity. Explicit partitioning of the inverse operators (Eq. (3)) ensures their positive-definiteness and provides a recursive algorithm (Eq. (20)) for computing the action of the BEC operator which has reasonably conditioned matrices on each iteration.

Presented results are also valid in the homogeneous anisotropic case, because the latter can be reduced to isotropic form by the appropriate coordinate transformation (e.g. Xu, 2005; Yaremchuk and Carrier, 2012). The obtained analytical expressions for the correlation functions (Eqs (1015)) can be useful in finding the BEC model parameters for the fields whose local decorrelation scales ρ do not change significantly at distances of the order of ρ.

In the more general inhomogeneous case analytical formulas for the matrix elements of B cannot be obtained, and inversion of the operator (Eq. (20)) has to be performed numerically. For the BEC models with M > 2 such inversion may encounter difficulties associated with the condition number of B, which grows exponentially with the number of model parameters. In view of the decomposition (Eq. (20)), however, this inversion can be performed consecutively by iterative solutions of M linear systems whose condition numbers are limited from above by the maximum eigenvalue of equation image.

In higher dimensions (n > 1) the polynomial BEC model can be further improved by introducing anisotropic inhomogeneous diffusion operators separately for each mode. The respective diffusion tensors can be adjusted using prior knowledge of the impact of the background fields on the error characteristics described by the corresponding spectral peak.

For the BEC spectra characterized by deep gaps between the peaks, the multi-scale approach of Li et al. (pers. comm. 2012) may prove to be more computationally effective, as it does not take into the account scale interactions, and adopts the additive BEC model. In the case when a pair of closely spaced peaks (Figure 2(a)) exists, the technique of Li et al. (pers. comm. 2012) can be generalized by introducing a two-parameter model to account for the additive BEC component at the corresponding scale.


Real negative roots provide limited freedom to controlling the shape of the BEC spectrum because in this case the poles of Eq. (7) are located outside the range κ > 0. Therefore, these poles just add a monotonously decaying function of k which has the largest impact on the long-wave part of B(k). Furthermore, spectral contribution of the negative roots can always be well approximated by a pair of complex roots ε2b2 ± 2ibε, where ε is a small number (Figure 3).

Figure 3.

An example of the normalized spectrum generated by adding two negative roots p1 = 1 and p2 = 4 to the spectrum in Figure 1 (z1 = .5 + 3i; z2 = .2 + 6i). Grey spectrum is the approximation obtained by replacing p1 and p2 with z3 = −1.46 + .01i. The approximation error is 5%.

Taking N = L − 2M negative roots −pn (pn > 0) into account supplements all the formulas with an extra summation (product) over these roots. In the following, the ‘negative-root generalizations' of the key formulas are listed. For clarity, we keep the numbering and abbreviate sums/products from the main text by {Σ} and {Π} respectively:

equation image(A3)
equation image(A4)
equation image(A8)
equation image(A9)

where equation image, and equation image:

equation image(A10)
equation image(A11)
equation image(A12)
equation image(A16)
equation image(A19)


This study was supported by the Office of Naval Research (Program element 0602435N).