An approach to adaptive ﬁltering with variable step size based on geometric algebra

Recently, adaptive ﬁltering algorithms have attracted much more attention in the ﬁeld of signal processing. By studying the shortcoming of the traditional real-valued ﬁxed step size adaptive ﬁltering algorithm, this paper proposed the novel approach to adaptive ﬁltering with variable step size based on Sigmoid function and geometric algebra (GA). First, the proposed approach to adaptive ﬁltering with variable step size based on geometric algebra represents the multi-dimensional signal as a GA multi-vector for the vectorization process. Second, the proposed approach to adaptive ﬁltering with variable step size based on geometric algebra solves the contradiction between the steady-state error and the convergence rate by establishing a non-linear function relationship between the step size and the error signal. Finally, the experimental results demonstrate that the proposed approach to adaptive ﬁltering with variable step size based on geometric algebra achieves better performance than that of the existing adaptive ﬁltering


INTRODUCTION
Adaptive filters adjust its weight coefficients according to the input signal and the current state, which has been widely applied in various fields, such as channel equalization [1], noise cancelation [2], acoustic echo cancelation [3,4], system identification [5]. The most common adaptive filtering algorithm includes the least mean square (LMS) [6], which is based on the mean square error (MSE) criterion. Due to its robustness and simplicity, the LMS algorithm has been widely used in many fields. Rastegarnia et al. [7] proposed the constrained distributed least mean square (DLMS) algorithm over the distributed estimation of the wireless sensor networks (WSNs), where the cost function is modified to consider the observation noise variance of each sensor. Tan et al. [8] proposed the workable data communication scheme by utilizing the hierarchical least mean square (HLMS) adaptive filter, which solves the restriction of scare LMS algorithm cannot obtain low steady-state error and fast convergence rate at the same time. Thus, the performance is degraded because of the trade-off between the steady-state error of the convergence rate.
To eliminate the trade-off, the variable step size adaptive filtering algorithms have been proposed [18,19]. The variable step size algorithm is based on the principle that improving the convergence speed with large step size at the beginning, and then reducing the steady-state error with small step size when the steady-state error reduced gradually to establish the functional relationship between the steady-state error and step size. Qin et.al [20] proposed the variable step size LMS (SVSLMS) adaptive filtering algorithm based on Sigmoid function, which overcomes the trade-off between the steady-state error and the convergence rate. Experimental results confirmed that the proposed SVSLMS algorithm achieves fast convergence rate and low residual error. However, the step size of this SVSLMS algorithm changes quickly in the steady state. It will affect the steadystate error of this algorithm. To solves this problem, Gao et.al [21] presented a novel SVSLMS algorithm, which establishes a new non-linear functional relationship between the step size and the error signal e(n). The new non-linear functional relationship is much simpler than the Sigmoid function and the error e(n) can be slightly changed near to zero. The simulation results demonstrate that the performance of the proposed SVSLMS algorithm outperforms the SVSLMS algorithm proposed in [20], which has less computational complexity. Although the steady-state error is reduced, the convergence rate and antiinterference of the proposed algorithm will be weakened. Both of the above algorithms in [20,21] derive the SVSLMS algorithm based on the Sigmoid function. Besides, Yan et.al [22] proposed a novel SVSLMS algorithm based on the hyperbolic tangent function, which improves the convergence speed and reduces the steady-state error of the adaptive filter when tracking the system changes. What is more, the proposed SVSLMS algorithm eliminates the disturbances of the existing independent noises by using the autocorrelation of the error signal e(n) (current state) and e(n − 1) (previous state). Liu et al. [23] presented the novel component-wise variable step size diffusion distribution (CVSS) algorithm to estimate specific parameters in sensor networks, which reduces the steady-state error and achieves a fast convergence rate than serval other LMS algorithms.
With the upcoming fifth-generation (5G) wireless communication system, the Internet of Things (IoT) becomes more and more important because 5G will become an important enable for the IoT. In addition, the development of 5G technology has also promoted the development of big data and 3D technologies [24], relay communications [25,26], vehicular ad hoc networks [27], cloud computing [28][29][30] and recommender system [31]. To improve the estimation performance of the diffusionbased LMS algorithm for WSN, Bin Saeed et al. [32] proposed a unified analytical framework for distributed SVSLMS algorithm and then expanded this analysis to the diffusion-based WSNs. Besides, IoT is an emerging trend in the field of information and communication technology. Its main challenge is to gather of data streams from millions of sensors in time. Giouroukis et al. [33] investigated representative and state-of-the-art algorithms to solve the scalability challenges in real-time and distributed sensor systems. However, it should be noted that the performance of the LMS-type algorithm will be degrade in the non-Gaussian situation. The performance of the adaptive filtering algorithm based on high-order statistics of the error signal outperform the LMS algorithm in some important applications [34][35][36]. The least mean kurtosis (LMK) [37] is one of the high-order moment adaptive filtering algorithms where its cost function is based on the negated kurtosis of the error signal and the kurtosis is defined by the fourth-order cumulant [38,39]. However, the fixed step size of the traditional LMK algorithm results in the conflict of the convergence rate and the step size. Meanwhile, the real-valued variable step size adaptive filtering algorithm cannot deal with the multi-dimensional signal well.
Fortunately, geometric algebra (GA) provides an effectiveness computing framework for multi-dimensional signal processing [40,41]. As a mathematical language, geometric algebra has been widely used in image and signal processing [42][43][44][45][46], physics [47] and computer vision [48,49]. GAAF algorithms are originally introduced in [50,51], which combined the GA theory with the adaptive filtering algorithm. Lopes et al. [50] exploit GA theory and geometric calculus (GC) to devise the novel GA least mean square (GA-LMS) adaptive filter. The performance of the GA-LMS algorithm is assessed by a 3D point-clouds registration problem, which includes a rotation estimation step. What is more, the adopted GA/GC framework allows the resulting filters to be easily applied to estimate rotors in higher dimensions. In order to further evaluate the performance of the GA-LMS algorithm, Al-Nuaimi et al. [51] applied GA-LMS algorithm to recover the 6-degree-of-freedom alignment of two point clouds related by a set of point correspondences. Based on the advantages of high-order statistics and geometric algebra, Wang et al [52,53] extends the traditional real-valued high-order statistics-based adaptive filtering algorithm into the GA space. The experimental results show that the proposed high-order statistics-based adaptive filtering algorithm achieves better performance in multi-dimensional signal prediction. Although the high-order statistics adaptive filtering algorithms based on geometric algebra improve the performance of the existing adaptive filtering algorithms in terms of the convergence rate and steady-state error, the step size still has a great influence on those adaptive filtering algorithms. In order to further improve the performance of GAAFs, this paper presents a novel approach to adaptive filtering with variable step size based on geometric algebra.
The main contributions of this paper are as follows. First, the multi-dimensional signal is represented as a multi-vector in the GA form where each component is mapped into the basis of GA space. Second, the GA/GC framework is employed to devise the SVSLMS and LMK adaptive filtering algorithms based on Sigmoid function in GA space, called GA-SVSLMS and GA-SVSLMK algorithms, which inherit its properties form GA and the traditional adaptive filtering algorithm. What is more, the use of GA/GC framework allows for applying the proposed variable step size algorithm in higher dimensions. Finally, this paper adopts several multi-dimensional signals to evaluate the performance of the proposed GA-SVSLMS and GA-SVSLMK algorithms. The experimental results demonstrate that the proposed approach to adaptive filtering with variable step size based on geometric algebra solves the trade-off between the convergence rate and the steady-state error. The rest of this paper is organized as follows. Section 2 introduces the basis of geometric algebra theory. Section 3 illustrates the proposed approach to adaptive filtering with variable step size based on geometric algebra in detail. Experimental results are presented in Section 4. Section 5 summarizes this paper.
To enhance readability, Table 1 summarizes the notation convention.

Geometric algebra
Geometric algebra (GA), also known as Clifford algebra, was proposed by William Kingdon Clifford [54]. Geometric algebra combines the quaternion algebra proposed by Hamilton with the exterior algebra proposed by Grassmann and provides a multi-dimensional unified computing framework which independent of the coordinate information [55,56]. For a set of orthogonal basis in ℝ n , the geometric product of the basis elements will generate 2 n basis members in n , that is, multivectors. Let n is the 2 n -dimensional GA space generated by an orthonormal basis of vectors e 1 , …,e n . The basis of n is defined as: Due to the multiplication operation in GA space is not commutative but is associative, the basis elements satisfy the following relations: In GA space, the core operation is geometric product which includes the inner product and the outer product. The geometric product of vector u and v is defined as: where u⋅v is the inner product and u⋅v = v⋅u, u∧v is the outer product and u∧v = -v∧u. Due to the outer product is noncommutative, the geometric product is non-commutative.
In GA space, the basic element is the multi-vector, which can be seen as the generalization of the complex and quaternion variable for higher dimensions. Multi-vector contains the scalar part and several other parts, such as bivector, trivector and so on. Therefore, a general multi-vector M in GA space is defined as: where the multi-vector is comprised of its s-vector part ⟨⋅⟩ s , s=0 is the scalar part, s=1 are the vectors, s=2 represent bivectors and s=3 are the trivectors.
In fact, any multi-vector M ∈ n can be decomposed into its blades via [57]: where M s is scalar valued, {e s } and {e s }, s=0,...,2 n -1 are basis of n .
In GA space, some common operations are shown below: • Scalar product:

The approach to adaptive filtering with variable step size
For the traditional LMS algorithm, its cost function and weight updating rule are given as: where e(n) is the error signal, w(n) is the weight vector, x(n) is the input signal vector, is the step size of the adaptive filter.
Due to the problem of the traditional real-valued LMS algorithm, Qin et. al [20] proposed the SVSLMS algorithm based on the Sigmoid function (SVSLMS) where the step size is variable. The Sigmoid function is defined as: According to formula (10), the non-linear relationship between the step size and the error signal e(n) is defined as: According to the variable step size formula, the contradiction between the step size and the convergence speed can be solved better.

3.1
The approach to LMS with variable step size based on Sigmoid function Lopes et al. [58] reformulated the traditional adaptive filtering algorithms under the framework of geometric algebra and developed a complete geometric algebra adaptive filters (GAAFs). The proposed GAAFs are generated by elaborating the underlying minimization problem from the perspective of the geometric algebra. Geometric calculus (GC), the extension of the GA which employing the same derivation formula regardless of the subalgebra of the variable, such as real valued, complex valued, and quaternion and so on. Taking advantage of GC and GA theory, the deterministic quadratic cost function is devised. According to the GA/GC framework, the cost function of the GA-LMS adaptive filtering algorithm is defined as [58]: in which D(i) is a multi-vector-valued signal and the input signal is In GA space, the GAAFs weight updating rule is given as: in which is the step size of the adaptive filtering algorithm, B is a matrix whose components are multi-vector entries.
According to the literature [57], the updating rule of the GA-LMS algorithm yields: in which B is setting as an identity matrix. Compared with the traditional real-valued LMS algorithm, GA-LMS can be applied to multi-dimensional signal processing, such as system identification scenario. Besides, it should be noted that the GA-LMS algorithm can be used to recover some traditional LMS adaptive filtering algorithms via algebraic isomorphisms, such as real LMS, complex LMS and quaternion LMS. The GA-LMS algorithm inherits properties from standard adaptive filtering algorithms and geometric algebra theory. For the GA-LMS adaptive filtering algorithm, its step size is fixed. In order to further improve the performance of the GA-LMS algorithm, the paper employs the Sigmoid function to establish the non-linear relationship between the step size and the error signal. The step size is defined as follows: where > 0, > 0. determines the range of values of this function and controls the shape of this function. Thus, the SVSLMS adaptive filtering algorithm based on Sigmoid function and geometric algebra (GA-SVSLMS) is proposed.

The approach to LMK with variable step size based on Sigmoid function
Wang et.al [52] presented a novel LMK adaptive filtering algorithm based on geometric algebra, which minimizes the cost function of negated kurtosis of the error signal in GA space. In the literature [52], the cost function of GA-LMK is defined as: According to the GA/GC framework, the updating rule of the GA-LMK algorithm is calculated as: where is the step size andP i = {|E(i)| 2 }, it can be calculated by the following recursive formula: in which is the forgetting factor. It can be seen from the literature [52] that the performance of GA-LMK algorithm is better than GA-LMS algorithm in processing non-Gaussian signals. In the case of small step size, GA-LMK algorithm achieves fast convergence rate and small steady-state error. However, the step size of GA-LMK algorithm is a fixed value, which must make a compromise between convergence speed and steady-state error. Therefore, this paper considers using Sigmoid function to further optimize the performance of GA-LMK algorithm. According to the Sigmoid function, the non-linear function between the step size and the error signal E(i) is given as: where > 0, > 0. determines the range of values of this function and controls the shape of this function. According to Equation (19), the variable step size LMK adaptive filtering algorithm based on Sigmoid function and geometric algebra (GA-SVSLMK) is proposed.

EXPERIMENTAL ANALYSIS
In this section, the proposed approach to adaptive filtering with variable step size based on geometric algebra is tested on one step ahead prediction problem. First, the proposed approach to adaptive filtering with variable step size based on geometric algebra and its corresponding real-valued adaptive filtering algorithms are compared. Then, the performance of the existing adaptive filtering algorithm based on geometric algebra is analysed on one step ahead prediction problem. The Lorenz attractor [59] is selected to evaluate the performance of the proposed GA-SVSLMS and GA-SVSLMK algorithms. Besides, to demonstrate the effectiveness and rationality of the proposed algorithm, a six-dimensional EEG signals are chosen to evaluate the performance of the proposed algorithms.

Comparison of SVSLMS and GA-SVSLMS algorithms
To validate the advantage of the geometric algebra framework, this section compares the real-valued SVSLMS with the proposed GA-SVSLMS algorithm. Figure 1a shows the performance of different algorithms for components of the 3D Lorenz attractor. Figure 1b,c shows the 3D estimated signal predicted by the SVSLMS and GA-SVSLMS algorithms.
As can be seen from above experimental results, the performance of the proposed GA-SVSLMS algorithm outperforms the traditional real-valued SVSLMS. The main reason can be attributed to the use of the GA/GC framework. The core element of GA/GC framework is the multi-vector, which is the basic element for the vector to expand into the high dimensional space. Multi-vector realizes the unified representation of different dimensions and different geometric objects. Meanwhile, multi-vector is used to express the geometric objects, which embodies the multi-dimensional unity and reserves the internal relationship of different components.

Comparison of SVSLMK and GA-SVSLMK algorithms
In this section, the real-valued SVSLMK and the proposed GA-SVSLMK algorithms are used to forecast 3D Lorenz attractor. The results of using different algorithms to track different components of the 3D Lorenz attractor is visualized in Figure 2a. The estimated signal predicted by SVSLMK and GA-SVSLMK algorithms are visualized in Figure 2b,c.
In this experiment, the real-valued SVSLMK algorithm treat the 3D Lorenz attractor as three independent components. However, there exists a complex non-linear and coupling relationship between the different dimensions of multi-dimensional signals, and the real-valued adaptive filtering algorithm fails to fully consider the relationship between different signal components which leads to the performance degradation. The GA-SVSLMK algorithm represents the multi-dimensional signal as a multi-vector in the GA form which captures the inner relationship between different signal components, resulting in quite better results than the traditional SVSLMK algorithm.

The performance of the variable step size adaptive filtering algorithm under different parameters
According to Equations (15) and (19), the performance of the variable step size adaptive filtering algorithm is affected by the coefficients and . Thus, this paper first discusses the effect of the coefficient and on the performance of the proposed algorithms.

Different coefficient
Compared with the traditional LMS-type algorithm, the step size of the proposed GA-SVSLMS algorithm is a variable. This section selects different coefficients and , and then analyses the performance of the GA-SVSLMS algorithm under different coefficients. Finally, a set of optimal coefficients and is selected to performed the following experiments. First, the value of coefficient is setting as 0.1, 0.01, 0.001, respectively. Figure 3a shows the performance of the proposed GA-SVSLMS algorithm under different . The estimated signal of the GA-SVSLMS algorithm under different coefficients is visualized in Figure 3b-d.
In order to further analyse the influence of the coefficient on the performance of GA-SVSLMS algorithm, the value of coefficient is selected as 0.2, 0.02, 0.002, respectively. Figure 4 shows the performance of the GA-SVSLMS algorithm.

Different coefficient
After analyzing the coefficient , this section continues to discuss the effect of coefficient on the performance of GA-SVSLMS algorithm. First, the coefficient is set as 0.02, 0.002, 0.0002, respectively. Figure 5 shows the performance of the GA-SVSLMS algorithm under different .
In Figure 5, the value of coefficient is set as 0.1. In the following experiments, the value of is 0.01, and then the effect of coefficient on the performance of the GA-SVSLMS algorithm is visualized in Figure 6. According to the above experiment results, the value of the coefficient and is chosen as: =0.1 and = 0.002.

GA-SVSLMK algorithm
In this section, the effect of the coefficients and on the GA-SVSLMK algorithm is discussed in detail. Different values are selected to analyse the performance of the GA-SVSLMK algorithm on one step ahead prediction problem. A set of coefficients are selected through several experiments which achieves the optimal performance of GA-SVSLMK algorithm.

Different coefficient
At the first time, the value of the coefficient is setting as 0.01, 0.001, 0.0001, respectively. Figure 7a shows the components of the 3D Lorenz attractor under different values. The estimated signal of different values is visualized in Figure 7b-d.
To discuss the performance of GA-SVSLMK algorithm in detail, the value of coefficient gradually increases to 0.04, 0.004, 0.0004, respectively. Figure 8 shows the prediction performance of the GA-SVSLMK algorithm under different .

Different coefficient
After discussing the influence of the coefficient on the performance of the GA-SVSLMK algorithm, this paper then analyses the prediction performance of the GA-SVSLMK  Figures 9 and 10. In Figure 9, the coefficient =0.0004. In Figure 10, the coefficient is set as 0.001. At this point, the GA-SVSLMK algorithm cannot achieve converge. Thus, the values of the coefficient and are chosen as: =0.001 and =0.0002.

Comparison of different adaptive filtering algorithms based on geometric algebra
In this section, the performance of the proposed approach to adaptive filtering with variable step size based on geometric algebra is compared with GA-LMS [58], GA-LMK [52], GA-LMF [53] and GA-LMMN [53] algorithms. The step size of GA-LMS, GA-LMK, GA-LMF and GA-LMMN are chosen to be: s = 9 × 10 −6 , k = 1.5 × 10 −6 , f = 1.5 × 10 −6 , and n = 1.5 × 10 −6 , respectively. Figure 11 shows the components of the 3D Lorenz attractor obtained by different adaptive filtering algorithms. The estimated signal of the 3D Lorenz attractor is visualized in Figure 12.
It can be seen from Figure 11, the GA-LMS algorithm requires much more iterations to track the actual signal than GA-SVSLMS algorithm. For GA-LMS algorithm, there is a contradiction between its steady-state error and the convergence speed. That is, the steady-state error is small but the con-vergence speed is slow in the case of small step size, while when the step size increases, the convergence speed is fast but the steady-state error increases. The high-order statisticsbased adaptive filtering algorithm, such as GA-LMK, solves this problem to an extent. However, the performance of high-order statistics-based adaptive filtering algorithm (GA-LMK, GA-LMF and GA-LMMN) will degrade as the step size increases. For the GA-LMS, GA-LMK, GA-LMF and GA-LMMN adaptive filtering algorithms, the step size is fixed, which makes the adaptive filtering algorithm to make a compromise between the steady-state error and the convergence speed. Compared with the fixed step size GA-LMS adaptive filtering algorithm, the proposed GA-SVSLMS solves the contradictory between the convergence rate and the stedy-state error. As can be seen from Figure 12, the estimated signal obtained by the proposed GA-SVSLMK is closer to the actual signal than GA-SVSLMS.
To further analyse the performance of the proposed variable step size adaptive filtering algorithm based on geometric algebra, this paper gives the learning curves of the different adaptive filtering algorithms. Figure 13 shows the learning curves of different adaptive filtering algorithms based on geometric algebra. Clearly, in Figure 13, compared with the GA-LMS algorithm, the GA-SVSLMS shows great improvement in terms of the mean absolute error. The mean absolute error of the GA-SVSLMK is much closer to the high-order statisticsbased adaptive filtering algorithms. However, the proposed GA-SVSLMK algorithm can effectively solve the problem that the performance of high-order statistics-based adaptive filtering algorithms become worse as the step size increases.

The multi-dimensional signal analysis
One of the advantages of geometric algebra is that it can use multi-vector to represent different dimensions and different geometric objects uniformly. By mapping the signal components of multi-dimensional signals to the basis of geometric

FIGURE 11
The tracking performance of different adaptive filtering algorithms algebra space, the intrinsic relationship between the signal components of different dimensions is well preserved. In order to further verify the performance of the proposed approach to adaptive filtering with variable step size based on geometric algebra in multi-dimensional signal processing, several experiments are performed in this section. The performance of the proposed GA-SVSLMS and GA-SVSLMK algorithms for six dimension signal are depicted in Figure 14. Considering the performance of the proposed GA-SVSLMS and GA-SVSLMK algorithm in practical application, the electroencephalogram (EEG) data [60] is selected in this section. The prediction performance of the variable step size adaptive filtering algorithm based on geometric algebra is depicted in Figure 15.

Computational complexity
This section analysed the computational complexity of the proposed GA-SVSLMS and GA-SVSLMK algorithm. The elapsed times of different adaptive filtering algorithms are shown in Table 2. From Table 2, we can clearly see that, owning to the high-order statistics, GA-LMK, GA-LMF and GA-LMMN algorithms have higher computational complexity. The proposed GA-SVSLMS algorithm effectively solves the trade-off between the steady-state error and the convergence speed, and its computational complexity is low.

FIGURE 13
The learning curves of different adaptive filtering algorithms

CONCLUSION
In this paper, the novel approach to adaptive filtering with variable step size based on geometric algebra is proposed, which formulates the multi-dimensional signal as a multi-vector in GA space. The proposed GA-SVSLMS and GA-SVSLMK algorithms show better at preserving the non-linear and coupling relationship between different components of the multidimensional signal, resulting in quite better results than traditional adaptive filtering algorithms. The experiments demonstrate that the proposed approach to adaptive filtering with variable step size based on geometric algebra solves the contradic-  tory between the steady-state error and the convergence speed which exists in the fixed step size adaptive filtering algorithm. Currently, the proposed approach to adaptive filtering with variable step size focus on the multi-dimensional signal prediction. Future work on the approach to adaptive filtering with variable step size will include applying it to more directions, such as WSNs, IoT, etc. On the other hand, how to reduce the computational complexity of adaptive filtering algorithm based on geometric algebra will be further discussed in the future work. It is expected that the proposed GA-SVSLMS and GA-SVSLMK algorithm will be a homogeneous and efficient tool in multidimensional signal processing.

ACKNOWLEDGEMENT
This work was supported by the National Natural Science Foundation of China under Grants 61771299, 61771322.