Recursive joint Cramér‐Rao lower bound for parametric systems with two‐adjacent‐states dependent measurements

National Natural Science Foundation of China, Grant/Award Numbers: 61673313, 61673317; National Key Research and Development Plan, Grant/Award Number: 2017YFB1301101 Abstract Joint Cramér‐Rao lower bound ( JCRLB) is very useful for the performance evaluation of joint state and parameter estimation ( JSPE) of non‐linear systems, in which the current measurement only depends on the current state. However, in reality, the non‐linear systems with two‐adjacent‐states dependent (TASD) measurements, that is, the current measurement is dependent on the current state as well as the most recent previous state, are also common. First, the recursive JCRLB for the general form of such non‐linear systems with unknown deterministic parameters is developed. Its relationships with the posterior CRLB for systems with TASD measurements and the hybrid CRLB for regular parametric systems are also provided. Then, the recursive JCRLBs for two special forms of parametric systems with TASD measurements, in which the measurement noises are autocorrelated or cross‐correlated with the process noises at one time step apart, are presented, respectively. Illustrative examples in radar target tracking show the effectiveness of the JCRLB for the performance evaluation of parametric TASD systems.


| INTRODUCTION
To assess the performance of parameter estimators, we can resort to the lower bound and the upper bound. They assess estimation performance from two different aspects. The lower bound puts a limit to the best estimation performance, whereas the upper bound [1] puts a limit to the worst estimation performance. For lower bound, the most well-known one is the Cramér-Rao lower bound (CRLB). For upper bound, a lot of work has been investigated. For example, a multi-innovation stochastic gradient algorithm and the corresponding parameter estimation errors upper bound were proposed in [2] for time-invariant stochastic systems. For systems with timevarying parameters, a finite data window stochastic gradient identification algorithm was proposed in [3], in which the minimum parameter estimation error upper bound was obtained by choosing the data window length. A desirable property for estimators is the consistency [4] if their estimation errors converge to zero in a certain sense. For example, a hierarchical least squares identification algorithm was proposed in [5] by decomposing a dual-rate system model into several subsystems model for dual-rate linear systems with noises. This algorithm is consistent and has low computational complexity. For general dual-rate sampled-data systems, a computationally efficient algorithm was proposed in [6] using hierarchical identification principle, in which the parameter estimates can converge to their true values. When the measurements of the systems are scarce, a gradient-based parameter identification method [7] was proposed and its convergence property was also analysed. advanced non-linear filters have been developed, for example, the unscented filter (UF) [10,11], quadrature Kalman filter (QKF) [12,13], cubature Kalman filter (CKF) [14][15][16], and so forth. All these non-linear filters aim to approximate the firsttwo moments used in the linear minimum mean-squared error (LMMSE) estimation. The UF approximates the first-two moments based on the unscented transformation of welldesigned sigma-points. The QKF calculates the first-two moments using the Gauss-Hermite quadrature integration rules, but the number of integral points increases exponentially as the dimension of state increases. The CKF transforms the integral problem into a summation problem using a thirddegree spherical-radial cubature rule, by which it can approximate the first-two moments needed in the LMMSE. In the recent years, another well-known non-linear filter is the particle filter (PF) [17,18]. The PF uses sequential Monte Carlo method to generate a large number of samples to characterize the posterior probability density. For the state estimation of special systems, for example, bilinear systems, the standard Kalman filter is not applicable. A state filtering method was developed in [19] by using delta operator to minimize the covariance matrix of state estimation errors for single-input-single-output and multiple-input-multiple-output bilinear systems. For linear systems with multistate delays, extended state Kalman filter will suffer from heavy computational burden. To overcome this, a highly computationally efficient state filter was proposed for such systems in [20] following the same idea of [19].
All the above estimators mainly deal with the regular nonlinear systems, in which the current measurement only depends on the current state. However, in many real applications, the non-linear systems are more complex [19,20]. For example, non-linear systems with two-adjacent-states dependent (TASD) measurements are also common. Actually, non-linear systems with autocorrelated measurement noises or with crosscorrelated process and measurement noises at one time step apart [21] are two typical cases of such systems. To handle these two cases, one way is to reformulate the measurement equation as a new measurement equation which is dependent on the current state as well as the most recent previous state. In many real applications, the whiteness assumption for noise as in the regular non-linear systems cannot be completely satisfied, where the measurement noise is either autocorrelated or cross-correlated with the process noise. In the radar tracking system, the measurement noises are always autocorrelated, which usually influences the maneuvering target tracking performance [22,23]. In the global navigation satellite system, suffering from signal outages and multi-path error, the measurement noise becomes autocorrelated [24][25][26]. Parameters estimation is an important subject in modern signal processing where the measurement noises involved is often autocorrelated [27,28]. The systems with dependent process and measurement noises is also very common in practice [29]. Particle filters were proposed in [30] for these cases in which the process and measurement noises are dependent. Some practical applications of models with dependent noises, for example, sensor fusion and econometrics, have been shown in [31]. To assess the performance of non-linear filters, the posterior CRLB (PCRLB) is proposed. In [32], a recursive approach to compute the PCRLB was developed for filtering of regular non-linear dynamic systems. To evaluate the performance for filtering of non-linear systems with TASD measurements, a recursive CRLB was developed in [33] for non-linear systems with the coloured noise, in which the first-order coloured measurement noise case is a typical TASD system, and a recursive PCRLB was proposed in [34] for the non-linear systems with cross-correlated process and measurement noises at one time step apart. In [35], a conditional PCRLB for TASD systems was proposed, which depends on the measurement history up to now. Because it relies on a specific implementation, it can be used to guide online sensor selection.
Actually, most discrete-time dynamic systems may incorporate some parameters which can be random or nonrandom. Joint state and parameter estimation (JSPE) has got a growing interest in various applications, such as target tracking [36], signal processing [37], sensor registration [38]. The main approaches for JSPE include joint filter [39,40], dual filter [41,42] expectation maximization method [43,44] and so forth. Joint filter schemes usually augment the parameters into a state vector and use various kinds of filters, for example, EKF, UKF, and so forth, to simultaneously estimate both. Dual filter schemes use two separate filters for state and parameter estimation. It decouples parameters and states and ignores the relevant information between them. Expectation maximization schemes solve the joint estimation problem by two iterative steps, in which the Estep assumes that the parameters are known and estimates the states, and the M-step uses the estimated states to identify the system parameters. To assess the performance for JSPE, hybrid CRLB (HCRLB) is introduced in [45][46][47][48][49]. A unified framework for HCRLB applied to the joint estimation of random and non-random parameters was summarized in [50]. Then the asymptotic tightness of the HCRLB was analysed in [51]. For the purpose of computational efficiency, the recursive form of HCRLB was proposed for the ground moving extended target tracking problem [52], and the more general form of it was obtained for the discrete-time systems involving unknown time invariant deterministic parameters in [53]. Furthermore, the HCRLB was extended to the non-linear systems with timevariant measurement parameters in [54].
Performance evaluation of JSPE for parametric TASD systems has not been studied yet. This article studies this problem and aims at proposing a recursive performance bound for the JSPE of parametric systems with TASD measurements. The contributions of this paper are as follows.
3. We present specific JCRLBs for two special cases of parametric TASD systems, including non-linear systems with autocorrelated measurement noises or with cross-correlated process and measurement noises at one time step apart.
The rest of this article is organized as follows. Section 2 formulates the JCRLB problem and presents the non-linear TASD systems with unknown deterministic parameters. Then, a recursive JCRLB to evaluate the performance of JSPE for these systems is proposed in Section 3. Section 4 develops the JCRLBs for two specific non-linear parametric systems. In Section 5, some illustrative examples in radar target tracking are provided to verify the effectiveness of the proposed JCRLB. Section 6 concludes this paper.

| PROBLEM FORMULATION
As we know, the classical CRLB is suitable for the estimation of non-random parameters, whereas the Bayesian CRLB is applicable to random parameters. In joint parameter estimation problem, one wishes to estimate an unknown joint parameter vector χ = (x 0 , θ 0 ) 0 given the measurement z, where x is a random parameter vector and θ is a non-random parameter vector. The joint Fisher information matrix ( JFIM) for χ is then where p(x, z|θ) is the joint conditional probability density function (PDF), Δ denotes the second-order derivative operator, that is, Δ b a ¼ ∇ a ∇ 0 b , and ∇ denotes the gradient operator. Let b χ ðzÞ be an unbiased estimator of χ. The JCRLB on the mean squared error (MSE) is defined as the inverse of the JFIM [49]: Consider the following general form of non-linear parametric systems with TASD measurements.
where θ x and θ z are unknown as the deterministic parameter vectors, the process noise hw k i and the measurement noise hv k i are mutually independent white noises with PDFs p(w k | β) and p(v k |γ), respectively, which depend on unknown deterministic parameter vectors β and γ, and the initial state x 0 is independent of the process and measurement noises with PDF p (x 0 |α), which depends on the unknown deterministic parameter α. The joint estimand (quantity to be estimated) consists of the system state x k ∈ R n and non-random parameter vector Unlike in regular parametric systems, the measurement z k in TASD systems depends on both the current state x k and the most recent previous state x k−1 . The main goal of this paper is to obtain a recursive JCRLB for filtering of non-linear parametric systems with TASD measurements.

| JCRLB
From Equations (1) and (2), the joint conditional probability distribution of is the joint conditional probability distribution of x k and z k given θ Definition 3 Define the JFIM J k about the joint estimand sequence χ k as where θ 0 is the true value of the unknown non-random parameter θ.

Lemma 1
The MSE of a joint unbiased estimate b χ k is bounded from below by the inverse of the J k [49] as Definition 4 Define J −1 k as the (n + m) � (n + m) right-lower block of ðJ k Þ −1 and J k as the JFIM about x k and θ,

Lemma 2
The MSE of a joint unbiased estimate b χ k is bounded from below by the inverse of J k [51,53] as We aim to obtain a recursive form to calculate J −1 k without manipulating the large matrix J k .

Theorem 1 Given the Fisher information matrix
the recursion from J k to J k+1 can be obtained as where and information J 0 can be obtained from the prior probability distribution p(x 0 |α) as Proof See Appendix 1.
Next, we discuss two commonly encountered cases in practice, including the additive Gaussian noise case and linear additive Gaussian noise case.

Corollary 1
Suppose that the discrete-time non-linear system (Equations (1) and (2)) is driven by additive Gaussian noises as where the noises hw k i and hv k i are mutually independent white Gaussian processes with zero mean and invertible covariance matrices Q k and R k , respectively. Then Equation (6) will be simplified to Corollary 2 Suppose that the discrete-time system (Equations (8) and (9)) is further reduced to a linear system with additive Gaussian noises as where the noises hw k i and hv k i are mutually independent white Gaussian processes with zero mean and invertible covariance matrices Q k and R k , respectively. Then Equation (10) will be simplified to where 0's are zero matrices with appropriate dimensions.
Proof See Appendix 3.

| Relationship with the PCRLB for TASD systems
Corollary 3 Suppose that the discrete-time system (Equations (1) and (2)) is reduced to the following nonparametric non-linear TASD system where the process noise hw k i and the measurement noise hv k i are mutually independent white noise sequences. The initial state x 0 is independent of the process and measurement noises. Then the recursion for J k in Theorem 1 will be reduced to where This is exactly the FIsM for non-parametric TASD systems proposed in [33,34]. Obviously, Equation (16) is a special case of the JCRLB in Theorem 1.

| Relationship with the HCRLB for regular parametric systems
In regular parametric systems, the current measurement z k only depends on the current state x k , that is, z k = h k (x k , v k , θ z ). This is a special case of Equation (2). For such systems, the likelihood PDF p(z k+1 |x k+1 , x k , θ) in Equation (3) will be reduced to p (z k+1 |x k+1 , θ). As a result, in Equation (6) we have Similarly, Then, Equation (6) will be simplified to LI ET AL. -225 The JFIM J k in Equation (5) with Equation (6) replaced by Equation (19) is exactly the FIM for regular parametric systems proposed in [53]. Therefore, Equation (19) is a special case of the JCRLB in Theorem 1.

| JCRLB FOR TWO SPECIAL TYPES OF NON-LINEAR SYSTEMS
The non-linear parametric systems, in which the measurement noises are either autocorrelated or cross-correlated with the process noises at one time step apart, are two special types of TASD systems. Specific JCRLBs for these two types of systems will be presented next.

| JCRLB for non-linear systems with autocorrelated measurement noises
Consider the following non-linear system: where θ x and θ z are unknown deterministic parameter vectors, the process noise hw k i is white, the measurement noise he k i is autocorrelated satisfying the following first-order autoregressive(AR) model [23].
where the driven noise hξ k−1 i is white, hw k i and hξ k−1 i are mutually independent, and both independent of the initial state x 0 as well. The difference of two adjacent measurements yields By treating z k as a pseudo measurement, we have the following TASD measurement equation where Obviously the pseudo measurement noise hv k i is white, and independent of hw k i and x 0 .
Applying Theorem 1 to the equivalent TASD system (Equations (20) and (24)), we can obtain the recursive JCRLB for non-linear systems with autocorrelated measurement noises.
Next we discuss some specific forms of the JFIM for nonlinear systems with autocorrelated measurement noises when the noises are Gaussian and the systems are linear. (20) and (22)), if w k ∼ N ð0; Q k Þ and ξ k ∼ N ð0; R k Þ, then Equation (6) in Theorem 1 will be simplified to

Theorem 2 For the non-linear system (Equations
Remark 2 Suppose that the non-linear system (Equations (20) and (22)) does not depend on any nonrandom parameters, that is, θ ∈∅. By applying Corollary 3, we can obtain the PCRLB for non-linear systems with autocorrelated measurement noises, which is the same as Corollary 3.5 in [33]. (20) and (22)) is further reduced to a linear system

Corollary 4 If the non-linear system (Equations
then Equation (25) in Theorem 2 will be further simplified to where 0's are zero matrices with appropriate dimensions.

| JCRLB for non-linear systems with noises cross-correlated at one time step apart
Consider the following non-linear system: where θ x and θ z are unknown deterministic parameter vectors, hw k i and he k i are zero-mean white noises cross-correlated at one time step apart [30] and independent of the initial state x 0 as well, satisfying where δ k,j−1 is the Kronecker delta function. As in [35], the following TASD measurement equation can be obtained and the pseudo measurement noise hv k i and hw k−1 i are mutually independent white noises, and the mean and covariance of hv k i are zero and respectively. Applying Theorem 1 to the equivalent TASD system (Equations (28) and (30)), we can obtain the recursive JCRLB for non-linear systems with cross-correlated process and measurement noises at one time step apart.
Next we discuss some specific forms of the JFIM for nonlinear systems with noises cross-correlated at one time step apart when the noises are Gaussian and the systems are linear. (28) and (29)), if w k ∼ N ð0; Q k Þ and e k ∼ N ð0; E k Þ, then Equation (6) in Theorem 1 will be simplified to

Theorem 3 For the non-linear system (Equations
Proof See Appendix 5. Remark 3 Suppose that the non-linear system (Equations (28) and (29)) does not depend on any non-random parameters, that is, θ ∈ ∅. By applying Corollary 3, we can obtain the PCRLB for non-linear systems with noises cross-correlated at one time step apart, which is the same as Theorem 4 in [34].

Corollary 5 If the non-linear system (Equations 28 and 29) is further reduced to a linear system,
then Equation (31) in Theorem 3 will be further simplified to where 0's are zero matrices with appropriate dimensions.

| ILLUSTRATIVE EXAMPLES
In this section, numerical examples in the radar target tracking are provided to demonstrate the effectiveness of the proposed recursive JCRLB for JSPE of non-linear parametric systems with TASD measurements. However, to the best of our knowledge, there does not exist any unbiased joint estimator for this problem. So we design the following numerical examples to show how hard it is to jointly estimate the target motion state and radar measurement biases. Consider a single target with coordinated turn motion in the two-dimensional plane [14,16,29,33].
where X k ¼ x k ; _ x k ; y k ; _ y k � � 0 is the state vector, T = 1 s is the sampling interval, ω = 2 • s −1 is the turning rate, w k is a zeromean white Gaussian process noise with covariance Q k = cov (w k ) = diag(qM, qM), q = 0.01 m 2 s −3 and A two-dimensional radar located at the origin is used to track the motion of the target ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi where z k+1 is the radar measurement vector consisting of the range measurement r k+1 and the bearing measurement θ k+1 , Δ r and Δ θ are range and bearing measurement biases, which are unknown non-random parameters with ground truth 20 m and 5 mrad, respectively, and he k+1 i is the measurement noise.

| Example 1: autocorrelated measurement noises
In this example, we assume that the measurement noise he k+1 i in Equation (37) is autocorrelated and governed by where I is a 2 � 2 identity matrix, hξ k i is a zero-mean white Gaussian driven noise with covariance R k = diagðσ 2 r ðξÞ; σ 2 θ ðξÞÞ. Moreover, hw k i and hξ k i are mutually independent, and both of them are independent of the initial state X 0 . The distribution We consider three different settings for σ r (ξ) and σ θ (ξ) in Table 1 Figures 1-4 show that the JCRLBs of position, velocity, range bias Δ r and bearing bias Δ θ increase as the covariance of the pseudo measurement noise hξ k i increases. From Equation (39), we know that the uncertainty of the pseudo measurement z � kþ1 is only dependent on the uncertainty of the white noise hξ k i owing to its independence of state x k+1 and x k and that the uncertainty of x k+1 and x k is fixed. So if the covariance R k increases, then the uncertainty of the pseudo measurement noise increases and the pseudo measurement z � kþ1 is more uncertain. Thus the JCRLB of the non-linear system gets larger when R k gets larger. That is, the larger the pseudo measurement noise level is, the more difficult it is to jointly estimate the target motion state and radar measurement biases. -229 5.2 | Example 2: cross-correlated process and measurement noises at one time step apart In this example, we assume that the measurement noise e k ∼ N 0; E k ð Þ in Equation (37) is cross-correlated with the process noise hw k i in Equation (35) at one time step apart. The cross-correlation between w k and e k+1 is We consider three different settings for σ r (e) and σ θ (e) in Table 2. Experiments over 1000 Monte Carlo runs are used to show how the proposed JCRLB can be used to evaluate the estimation performance in this radar target tracking example with cross-correlated process and measurement noises at one time step apart.
According to Equation (30), we can obtain the covariance of the equivalent TASD measurement noise for this example as Figures 5-8 show that the JCRLBs of position, velocity, range bias Δ r and bearing bias Δ θ increase as the covariance of measurement noise he k i increases. From Equations (30) and (40), we know that if the covariance E k+1 of the measurement noise he k i increases while U k and Q k are fixed, then the covariance R k+1 of the equivalent TASD measurement noise hv k i increases and the equivalent TASD measurement is more uncertain. Thus the JCRLB of the non-linear system gets larger when E k+1 gets larger. In other words, the larger the measurement noise level gets, the more difficult it is to estimate the target motion state and radar measurement biases. We have developed a recursive JCRLB for JSPE of dynamic nonlinear parametric systems with TASD measurements, that is, the current measurement depends on both the current state and the most recent previous state. Its connections with the PCRLB for the systems with TASD measurements and the HCRLB for regular parametric systems, in which the current measurement only depends on the current state, have been studied as well. It is found that both the PCRLB for TASD systems and the HCRLB for regular parametric systems are special cases of the JCRLB for the TASD parametric systems. The main difference between them is due to the use of different likelihood functions. Meanwhile, specific and explicit forms of JCRLBs for commonly encountered Gaussian noise case and linear Gaussian case have been presented. The recursive JCRLBs for two typical forms of the systems with TASD measurements, that is, systems with autocorrelated measurement noises or cross-correlated process and measurement noises at one time step apart, have also been investigated. Moreover, simplified and explicit forms of the recursive JCRLBs have been obtained for these two types of systems for the Gaussian noise case and the linear Gaussian case as well. -231 Similarly, decompose χ k+1 as χ k+1 = [(x k−1 ) 0 , (x k ) 0 , (x k+1 ) 0 ,