Iterative parameter estimation methods for dual-rate sampled-data bilinear systems by means of the data ﬁltering technique

This paper considers the iterative parameter estimation for a dual-rate sampled-data bilinear system with autoregressive moving average noise. Through combining the auxiliary model identiﬁcation idea with the data ﬁltering technique, this paper derives two ﬁltering auxiliary model gradient-based iterative algorithms by using two different ﬁlters. The key is to construct an auxiliary model for predicting the unavailable outputs, and to transform the dual-rate bilinear system identiﬁcation model into two sub-identiﬁcation models. Finally, an auxiliary model gradient-based iterative (AM-GI) algorithm is presented for comparison. The simulation results indicate that the proposed algorithms are effective for identifying the dual-rate sampled-data bilinear systems, and can generate more accurate parameter estimates and have a higher computational efﬁciency than the AM-GI algorithm.


INTRODUCTION
Nonlinear systems widely exist in various practical problems, and have many characteristics significantly different from linear systems [1][2][3]. The modelling, parameter identification and control of nonlinear systems are difficult because of their complexity [4][5][6]. Bilinear systems are a class of ideal nonlinear system models with a simple model structure in the form, whose models are a generalization of linear system models [7,8]. Due to the special structure of bilinear system models, they can describe a class of practical process more accurately than the linear system models [9,10]. Therefore, bilinear and nonlinear models have been widely used in the fields of engineering [11][12][13][14][15]. The parameter estimation of bilinear systems has received extensive attention in the past several decades and many parameter estimation methods have been developed [16][17][18]. For example, Gibson et al. proposed a maximum likelihood parameter estimation method for a bilinear system [19]; according to the multi-innovation identification theory, Meng presented a new stochastic gradient algorithm for bilinear systems with white noises [20]; based on the data filtering technique, a filtering least squares-based iterative algorithm has been developed for parameter estimation of bilinear systems with coloured noises [21]; Zhang  estimation algorithm for bilinear systems using the hierarchical identification principle [22,23]. In addition, transforming the bilinear system into an equivalent linear model is also an effective method to deal with the identification problem of bilinear systems, and many identification methods based on the model conversion have been developed, such as the observer/Kalman filtering based identification algorithm [24] and the intersection subspace algorithm [25]. Unfortunately, almost all of these methods assumed that the input-output data of bilinear systems are available at every sampling instant, that is, the outputs and the inputs of the bilinear systems have the same sampling rate. When the bilinear systems have dual-rate input-output data, these algorithms may be invalid. Motivated by these problems, this paper focuses on the identification of bilinear systems with dual-rate input-output data.
In the practical industrial process, there exist many multi-rate sampled-data systems because of the limitation of the measurement technology or any other hardware constraints, such as the production of yellow rice wine or the polymer [26,27]. Multirate sampled-data systems have been studied extensively in the aspects of signal processing, control theory and system identification and modeling [28,29]. A dual-rate sampled-data system, whose output sampling period is an integer multiple of the input updating period, is the simplest multi-rate sampled-data system [30]. The identification of dual-rate sampled-data systems has been paid great attention. For instance, Chen et al. developed a bias compensation recursive least squares algorithm for dualrate rational systems [31]. The main tools used for dealing with the dual-rate sampled-data systems are the polynomial transformation technique and the lifting technique, which can transform the original system into a dual-rate system containing all the available data. However, the dual-rate sampled-data bilinear model has a complex structure, and cannot be transformed into a system containing all the available data. Therefore, the polynomial transformation technique and the lifting technique cannot be applied to dual-rate sampled-data bilinear system identification.
The auxiliary model identification idea is useful for identifying systems containing the unmeasurable variables in the information vectors [32], and can be utilized for the dual-rate sampled-data bilinear system identification. The auxiliary model identification idea is to construct an auxiliary model by using the estimated parameters and to replace the unavailable outputs with the outputs of this auxiliary model [33]. Different from the polynomial transformation technique and the lifting technique, the auxiliary model-based method can keep the parameters to be estimated unchanged, and has been widely used for the dualrate sampled-data system identification. Ding et al. presented a filtering and auxiliary model based recursive least squares identification algorithm for dual-rate systems with time-delays [34].
The previous work [32,35,36] mainly studied the singlerate sampled-data bilinear system identification, the auxiliary model-based algorithms proposed in [33] and [34] are recursive methods for identifying the single-rate or dual-rate sampleddata linear systems. Different from the previous methods, this paper focuses on the iterative parameter identification problems of dual-rate sampled-data bilinear systems with coloured noises by combining the auxiliary model identification idea with the data filtering technique. The parameter estimation of dual-rate sampled-data bilinear systems with coloured noises is more difficult than linear systems because of the complex structure of bilinear systems. The main contributions are as follows.
• The difficulty of identification for the dual-rate sampled-data bilinear systems is that some outputs are unavailable. According to the auxiliary model identification idea, an auxiliary model is constructed by using the estimated parameters and the unavailable outputs are replaced with the outputs of this auxiliary model. • By means of filtering the dual-rate input-output data of the bilinear system with two different filters, obtain two different identification models and derive a half-filtering auxiliary model gradient-based iterative (HF-AM-GI) algorithm and a filtering auxiliary model gradient-based iterative (F-AM-GI) algorithm. • For comparison, an auxiliary model gradient-based iterative (AM-GI) algorithm is presented. The proposed HF-AM-GI algorithm and F-AM-GI algorithm have a higher computation efficiency and parameter estimation accuracy than the AM-GI algorithm.
The rest of this paper is organized as follows. Section 2 simply derives the identification model for dual-rate input-output bilinear systems with coloured noises. Sections 3 and 4 present an HF-AM-GI identification algorithm and an F-AM-GI identification algorithm by using the data filtering technique. Section 5 gives an AM-GI identification algorithm for comparison. Section 6 provides some numerical examples to validate the effectiveness of the proposed algorithms. Finally, some concluding remarks are made in Section 7.

SYSTEM DESCRIPTION AND IDENTIFICATION MODEL
Let us introduce some symbols for convenience. "R =: S " or "S := R" stands for "R is defined as S ". 1 n represents an ndimensional column vector whose entries are all 1. The superscript T denotes the matrix/vector transpose. s −1 symbolizes a unit backward shift operator: The bilinear system can be described by a state space model structure, and also can be expressed as an input-output representation form. The difficulty of identifying the state space bilinear system is that its model structure contains the products of the states and inputs. According to the derivation process in [32], this paper eliminates the state variables in the bilinear state space system with the observability canonical form, and obtains the following input-output bilinear system with the output-error type model structure: where u(t ) ∈ ℝ denotes the system input, y(t ) ∈ ℝ is the system output disturbed by a noise, w(t ) ∈ ℝ is a noise term with zero mean. A(s −1 ), B(s −1 ), C (s −1 ), and D(s −1 ) are the polynomials with respect to the unit backward shift operator s −1 , and can be described as A(s −1 ) := 1 + a 1 s −1 + a 2 s −2 + ⋯ + a n s −n , a i ∈ ℝ, In the dual-rate sampled-data bilinear systems, the input sampling period is assumed to be T , and the output sampling period is hT (h ⩾ 2 is an integer). Then the input data have a quicker sampled rate than the output data, that is, all the input data {u(qT ) : q = 1, 2, 3, …} and scarce output data {y(hqT ) : q = 1, 2, 3, …} are available at each sampling instant hT , and the missing output data {y(hqT + jT ) : j = 1, 2, … , h − 1} are not available. For convenience, the input data u(qT ) and the output data y(hqT ) are simply rewritten as u(q) and y(qh). The noise term w(t ) in (1) may be a white noise process, an autoregressive (AR) process, a moving average (MA) process or an ARMA process. Without loss of generality, this paper considers w(t ) as an ARMA process, that is, w(t ) := where v(t ) ∈ ℝ is a white noise process with zero mean, E (s −h ) and F (s −h ) are the polynomials in s −h and E (s −h ) := 1 + e 1 s −h + e 2 s −2h + ⋯ + e n e s −n e h , e i ∈ ℝ, Replacing t in (1) with qh and defining an intermediate variable (qh), the input-output bilinear system can be rewritten as where The objective of this paper is to develop auxiliary model gradient-based iterative algorithms using the data filtering technique for estimating the unknown parameters According to the above definitions, Equation (4) can be rewritten as Equation (7) is the noise identification model. From (3), we have Substituting (7) and (5) into (2) gives = T (qh) + w(qh) (10) Equation (13) is the identification model for the dual-rate input-output bilinear system with the output-error type model structure in (2), and the parameter vector contains all the parameters to be estimated, that is, the parameters a i , b i , c i and d i of the system model and the parameters e i and f i of the noise model. The proposed algorithms in this paper are based on this identification model in (13). Many identification methods are derived based on the identification models of the systems [37][38][39][40] and can be used to estimate the parameters of other linear systems and nonlinear systems [41][42][43][44][45][46][47] and can be applied to fields [48][49][50][51][52][53] such as chemical process control systems.

THE HALF-FILTERING AUXILIARY MODEL GRADIENT-BASED ITERATIVE ALGORITHM
In general, we can see from (2) that the output data y(qh) involve the coloured noise w(qh), which has a negative influence on the parameter estimation accuracy. Using a filter to screen the input-output data of systems, new identification models are obtained. Therefore, a filter L 1 (s −h ) = E (s −h ) is adopted to filter the dual-rate sampled input-output data, and a half-filtering auxiliary model gradient-based iterative (HF-AM-GI) algorithm is derived for the dual-rate sampled-data bilinear system.

New sub-identification models
For the dual-rate input-output bilinear system in (2), multiplying the both sides of (2) by L 1 (s −h ) gives Define the filtered output y f (qh) and the filtered information vector f (qh) as Then Equation (14) can be rewritten as From (15), we can see that the dual-rate bilinear system with ARMA noise can be transformed into a dual-rate bilinear system with MA noise, which has a simpler noise model structure.
Because E (s −h ) is to be identified and unknown, the presented algorithm can be implemented through the iterative scheme. Substituting (8) into (15) gives where Notice that both the system identification model in (16) and the noise identification model in (6) contain the parameter vector f . Define an intermediate variable w 1 (qh) as Then Equations (16) and (17) are the new identification models of bilinear systems filtered by the filter E (s −h ).

Derivation of the HF-AM-GI algorithm
Collect the dual-rate sampled input-output data {u (1), u(2), …, u(lh), y(h), y(2h), … , y(lh)}, where lh ≫ 4n + n e + n f − 1 denotes the data length. Define the stacked filtered output vector Y f (lh), the stacked filtered information matrix Ω(lh), the stacked information matrices Ψ e (lh) and Ψ f (lh), and the stacked noise vectors W (lh) and W 1 (lh) as Let ‖X ‖ 2 := tr[X X T ]. According to the above definitions and the identification models in (16) and (17), we define the following two criterion functions Let̂k andê k be the estimates of and e at iteration k, where k = 1, 2, 3, … is an iteration variable. Minimizing J 1 ( ) and J 2 (e) and using the negative gradient search givê where 1,k and 2,k are the iterative step-size. From (17), we have Substituting (20) into (19) giveŝ Notice that the information vector e (qh) in Ψ e (lh) and the stacked noise vector W (lh) contain the unknown noise terms w(qh − jh), and the information vector f (qh) in Ψ f (lh) involves the unknown noise terms v(qh − jh), so they are unknown. Besides, the filter E (s −h ) is related to the unknown parameters e i , so it is impossible to compute the filtered variables y f (qh) and f (qh), that is, the filtered output vector Y f (lh) and the filtered information vector (qh) in Ω(lh) are unknown. Equations (18) and (21) cannot computêk andê k directly.
The convergence of the identification algorithms is very important and relies on some assumptions or conditions. Some assumptions and conclusions about the convergence of the parameter estimation of the HF-AM-GI algorithm are given in the following.
(A1) Assume that the orders n, n e and n f of the system are known and u(qh) = 0, y(qh) = 0 and v(qh) = 0 as q ⩽ 0. In practice, we can determine the degrees by using the correlation analysis of the observation data.
(A2) Assume that the system is controllable and observable, and the input-output data are bounded.
(A3) On identification, parameter identifiability requires the persistent excited conditions, that is, the input u(qh) is a persistently excited signal.
(A4) In order to guarantee the convergence of the iterative parameter estimates, all the eigenvalues of matrices should be located inside a unit circle. That is to say, one conservative choice of the iterative step-sizes 1,k and 2,k should satisfy (46) and (47).
Under these conditions, the parameter estimates given by the HF-AM-GI algorithm will converge to their true values as the iteration k and the data length approach infinity.
The computational efficiency of the algorithm can be evaluated by the number of multiplications (including divisions) and additions (including subtractions). One multiplication or one addition is called a flop, which means the floating point operations. The sum of the flops is the computational efficiency of the algorithm. The computational efficiency of the HF-AM-GI algorithm is shown in Table 1, and we define n 1 and n 0 in Tables 1-3 as n 1 := 4n − 1 + n f and n 0 := n 1 + n e .

Different from Section 3, this section uses a filter
to filter the dual-rate sampled input-output data, and a filtering auxiliary model gradient-based iterative (F-AM-GI) algorithm is derived for the dual-rate sampled-data bilinear system.

New sub-identification models
Define the filtered output y m (qh) and the filtered information vector m (qh) as Multiplying the both sides of (2) by L 2 (s −h ) gives From (48), we can see that the dual-rate bilinear system with ARMA noise can be transformed into a dual-rate bilinear system with white noise. Similarly, the filter L 2 (s −h ) is unknown, and the proposed algorithm with this filter can be implemented by the iterative scheme.
Substituting (8) into (48) gives Equations (49) and (7) are the new identification models of bilinear systems filtered by the filter L 2 (s −h ).

Derivation of the F-AM-GI algorithm
Collect the dual-rate sampled input-output data {u(1), u(2), …, u(lh), y(h), y(2h), … , y(lh)}, and define the stacked filtered output vector Y m (lh), the stacked output vector Y (lh), the stacked filtered information matrix Φ m (lh), the stacked information matrices Ψ(lh) and Φ s (lh), and the stacked noise vector W (lh) as Based on the identification models in (49) and (7), we define the following two criterion functions Let̂k and̂k be the estimates of and at iteration k. Minimizing J 3 ( ) and J 4 ( ) and using the negative gradient search giveˆk where 3,k and 4,k are the iterative step-size. From (10), we have Substituting (52) into (51) giveŝ Notice that the information vector (qh) in Ψ(lh) contains the unknown noise term w(qh − ih), and the information vector (qh) in Φ s (lh) contains the unmeasured term (qh − j ), so they are unknown. Besides, the filter L 2 (s −h ) is related to the unknown parameters e i and f i , and the filtered variables y m (qh) and m (qh) are unknown, so are the stacked filtered output vector Y m (lh) and the stacked filtered information matrix Φ m (lh). Equations (50) and (53) cannot computêk and̂k directly.
, we can obtain their estimatesŷ m,k (qh) and̂m ,k (qh): Replacing Y m (lh) and Φ m (lh) in (50) with their estimateŝ Y m,k (lh) andΦ m,k (lh), and Ψ(lh), Φ s (lh) and in (53) with their estimatesΨ k (lh),Φ s,k (lh) and̂k −1 , we can obtain the filtering auxiliary model gradient-based iterative (F-AM-GI) identification algorithm for the dual-rate sampled-data bilinear systems: The procedures of computinĝk and̂k by the F-AM-GI algorithm are listed in the following.
The computational efficiency of the F-AM-GI algorithm is shown in Table 2.

THE AUXILIARY MODEL GRADIENT-BASED ITERATIVE ALGORITHM
In order to show the advantages of the proposed HF-AM-GI and F-AM-GI algorithms, this section gives the auxiliary modelbased gradient iterative identification algorithm for comparison.
Collect the dual-rate sampled input-output data {u (1) According to the identification model in (13), we define the criterion function J 5 ( ) as Let̂k := [̂T k ,̂T k ] T be the estimates of = [ T , T ] T at iteration k. Minimizing J 5 ( ) and using the negative gradient search, we can obtain the auxiliary model gradient-based iterative (AM-GI) identification algorithm for the dual-rate sampled-data bilinear systems: The procedures of computinĝk by the AM-GI algorithm are listed in the following.
In order to make it clear, we take an example. Assume that system orders n = 5, n e = 5, n f = 5 and the data length lh = 4000 with h = 2, then we have N 1 = 3192630, N 2 = 3146490, N 3 = 3790870, that is, the difference between the total flops of the HF-AM-GI algorithm and the AM-GI algorithm at each iteration is N 3 − N 1 = 3790870 − 3192630 = 598240, and the difference between the total flops of the F-AM-GI algorithm and the AM-GI algorithm at each iteration is N 3 − N 2 = 3790870 − 3146490 = 644380. The algorithm with higher computational efficiency means the algorithm has lower flops. Obviously, the HF-AM-GI algorithm and the F-AM-GI algorithm have a higher computational efficiency than the AM-GI algorithm.

EXAMPLES
Example 1. Consider a dual-rate input-output bilinear system with h = 2: y(qh) = (qh) + w(qh), The parameter vector to be estimated is given by   = [a 1 , a 2 , b 1 , b 2 , c 1 , c 2  In simulation, the input {u(t )} is taken as a persistent excitation sequence with zero mean and unit variance. {v(t )} is an uncorrelated noise sequence with zero mean and variance 2 = 2.00 2 , 2 = 5.00 2 and 2 = 8.00 2 , respectively. Assuming that the data length lh = 6000, and the outputs {y(2), y(4), … , y(6000)} are available, while {y(1), y(3), … , y(5999)} are unmeasurable. To show the advantages of the proposed algorithms, the AM-GI algorithm is applied to estimate the parameters of this system for comparison. The AM-GI estimates and errors with different noise variances are shown in Table 4, where := ‖̂k − ‖∕‖ ‖.
From the simulation results in Table 4, we can see that the AM-GI algorithm can give accurate parameter estimates for bilinear systems when the noise variance is small, and the parameter estimation errors of the AM-GI algorithm become large as the noise variance increase. Example 2. This example takes the simulation model and simulation conditions in Example 1.
The parameter vector = [a 1 , a 2 , b 1 , b 2 , c 1 , c 2 , d 2 , e 1 , f 1 ] T contains nine parameters. Here, we transform the dual-rate bilinear system identification model into two subsystems by using different filters. When the filter L 1 (s −h ) = E (s −h ) is adopted to filter the dual-rate sampled input-output data, two parameter vectors = [a 1 , a 2 , b 1 , b 2 , c 1 , c 2 , d 2 , f 1 ] T and e = [e 1 ] of the two subsystems contain eight parameters and one parameter, respectively. Similarly, when the filter is adopted, two parameter vectors = [a 1 , a 2 , b 1 , b 2 , c 1 , c 2 , d 2 ] T and = [e 1 , f 1 ] T of the two subsystems contain seven parameters and two parameters, respectively. In other words, these subsystems have lower dimensions than the bilinear system. Then, the HF-AM-GI algorithm and the F-AM-GI algorithm are applied to identify these subsystems, respectively.
The parameter estimates and their errors with different noise variances are shown in Tables 5 and 6. For comparison, the estimation errors versus k with different algorithms are plotted in Figures 1-3.
Example 3. For the dual-rate sampled-data systems, the polynomial transformation technique and the lifting technique are very useful tools. For example, by using the polynomial transformation technique, a maximum likelihood forgetting factor stochastic gradient identification algorithm was proposed for the identification of Hammerstein dual-rate systems [54]; based on the lifting technique, a decomposition based recursive least squares algorithm was developed for general dualrate sampled-data systems [55]. However, due to the complex structure of the dual-rate sampled-data bilinear model, the polynomial transformation technique and the lifting technique is not applicable, so do the identification methods in [54,55].
The auxiliary model based recursive least squares (AM-RLS) algorithms developed in [34,56] are mainly used for the parameter estimation of dual-rate linear systems. For comparison, this example extends the AM-RLS algorithm to study the parameter estimation of dual-rate bilinear systems. Taking the simulation model and simulation conditions in Example 1, the AM-RLS parameter estimates and their errors with different noise variances 2 are shown in Table 7 and Figure 4. From the simulation results in Tables 4-7 and Figures 1-4, we can draw the following conclusions.
• The HF-AM-GI algorithm and the F-AM-GI algorithm can generate smaller estimation errors than the AM-GI algorithm and the AM-RLS algorithm when the noise variance level is large-see Tables 4-6. • Under high noise variance levels, the HF-AM-GI algorithm and the F-AM-GI algorithm have a faster convergence rate than the AM-GI algorithm-see Figures 1-3. • The parameter estimation errors of the three algorithms become small as the noise variance decreases-see Tables 4-7. • Compared with the AM-RLS algorithms in [34,56], the iterative algorithms proposed in this paper can make full use of the input and output data, and have a better parameter identification accuracy-see Tables 4-7.

CONCLUSIONS
Based on the auxiliary model identification idea, this work presents a half-filtering auxiliary model gradient-based iterative (HF-AM-GI) algorithm and a filtering auxiliary model gradientbased iterative (F-AM-GI) algorithm for the dual-rate bilinear systems with ARMA noises. Compared with the auxiliary model gradient-based iterative (AM-GI) algorithm, the HF-AM-GI algorithm and the F-AM-GI algorithm have a higher computational efficiency due to the data filtering. Besides, the two proposed algorithms have faster convergence rate and higher parameter estimation accuracy than the AM-GI algorithm when the noise variance level is high. The simulation results show that the proposed algorithms can identify the dualrate bilinear systems well and can give accurate parameter estimates for the dual-rate bilinear systems in the stochastic frame work. The proposed iterative parameter estimation methods for dual-rate sampled-data bilinear systems by means of the data filtering technique in this paper can be extended to other linear and nonlinear stochastic systems with coloured noises [57][58][59][60][61] and can be applied to other control and schedule areas such as the information processing and transportation communication systems [62][63][64][65][66][67][68][69] and so on.