Non-linear optimal control for multi-DOF electro-hydraulic robotic manipulators

: A non-linear optimal (H-infinity) control approach is proposed for the dynamic model of multi-degree-of-freedom (DOF) electro-hydraulic robotic manipulators. Control of electro-hydraulic manipulators is a non-trivial problem because of their non-linear and multi-variable dynamics. In this study, the considered robotic system consists of a multi-link robotic manipulator that receives actuation from rotary electro-hydraulic drives. The article's approach relies first on approximate linearisation of the state-space model of the electro-hydraulic manipulator, according to first-order Taylor series expansion and the computation of the related Jacobian matrices. For the approximately linearised model of the manipulator, a stabilising H-infinity feedback controller is designed. To compute the controller's gains, an algebraic Riccati equation is solved at each time-step of the control algorithm. The global stability properties of the control scheme are proven through Lyapunov analysis. The proposed control method retains the advantages of typical optimal control, i.e. fast and accurate tracking of the reference setpoints under moderate variations of the control inputs.


Introduction
Electro-hydraulic actuation systems have been widely used in robotic manipulators, in construction machinery and in aircrafts [1][2][3]. Nowadays, manipulators and vehicle receiving hydraulic actuation constitute a huge global industrial sector [4][5][6]. Hydraulic manipulators are favoured in industry, mining, agriculture and construction work where large actuation forces are required [7][8][9]. For instance, in 2016 the construction business alone sold 0.7 million units of construction machines, while in 2019 the sales of robotic manipulators, including also hydraulic manipulators, reached the level of 0.4 million units. Hydraulic actuation in robotic manipulators exhibits specific technical advantages, such as large force and torque, fast response, small size-to-power ratio, reliability and low manufacturing cost [10][11][12]. Hydraulic heavyduty manipulators are irreplaceable for heavy workpiece handling. They improve the efficiency of fine manipulation of heavy loads, thus ensuring production, safety and reducing labour costs [13][14][15]. Comparing against electric actuators, hydraulic actuators have the advantage of producing high power while having a smaller size [16,17]. Minimisation of energy consumption is an issue that has to be taken into account in the development of such robotic systems [18,19]. Furthermore, high precision motion is a vital functionality for electro-hydraulic robotic systems [20,21].
On the other side, the control of electro-hydraulic robotic manipulators remains a challenging problem that exhibits a higher degree of complexity than the control of electrically actuated manipulators. This is because electro-hydraulic manipulators have complicated non-linear dynamics, and are subject to modelling uncertainties (such as friction, internal leakage, dead bands in valves operation and control saturation). Besides, the functioning of electro-hydraulic manipulators is subject to several parametric variations, such as uncertain or varying loads. Heavy loads lead to significant rigid-flexible coupling characteristics and mechanical deformation in multi-degree-of-freedom (DOF) hydraulic manipulators. For this reason, the design of reliable controllers for hydraulic manipulators is acknowledged to be a demanding task. Various control methods have been developed for electro-hydraulic robots. Most of the results on the control of these manipulators exclude actuator dynamics from the robots' dynamic behaviour. However, actuator dynamics affects also the dynamics of the entire robotic system and the stability properties of the robots' control loop [22,23]. The design of control and actuation schemes for hydraulic manipulators can be improved if rotary hydraulic actuators are used in place of linear hydraulic actuators [24][25][26]. Rotary vane actuators, functioning as rotational drives, provide rotational movements directly because they are constructed as joints and actuators in one. Nevertheless, these actuators may exhibit internal leakage and high friction. Therefore, the controller design for hydraulic manipulators should compensate for such perturbations too.
There is ongoing research on the non-linear control problem of hydraulic manipulators in several engineering tasks [7,[27][28][29][30]. In this paper, a novel non-linear optimal control method is developed for multi-DOF electro-hydraulic robotic manipulators [1]. First, the dynamic model of the electro-hydraulic robotic manipulator undergoes approximate linearisation around a temporary operating point which is updated at each time-step of the control method. This time-varying operating point is determined by the present value of the robot's state vector and by the last sampled value of the control inputs vector. The linearisation process relies on firstorder Taylor series expansion and on the computation of the related Jacobian matrices [31][32][33]. The modelling error which is due to the truncation of higher-order terms in the Taylor series expansion, is considered to be a perturbation which is asymptotically compensated by the robustness of the control method. For the approximately linearised model of the robot, an H-infinity (optimal) feedback controller is designed.
The proposed H-infinity controller provides the solution of the optimal control problem for the dynamic model of the electrohydraulic robotic manipulator under model uncertainties and external perturbations. It represents a min-max differential game taking place between (i) the controller which tries to minimise a quadratic cost function of the state vector's tracking error, (ii) the model uncertainty and external perturbation terms which try to maximise this cost function [1,34] of the stabilising feedback controller, an algebraic Riccati equation is being repetitively solved at each time-step of the control algorithm [35,36]. The global stability properties of the control algorithm are proven through Lyapunov analysis. First, it is demonstrated that the control scheme satisfies the H-infinity tracking performance criterion, which signifies elevated robustness against model uncertainty and external perturbations [1,37]. Moreover, under moderate conditions, it is proven that the control loop is globally asymptotically stable. Additionally, to implement state estimation-based control without the need to measure the entire state vector of the robotic system, the H-infinity Kalman filter is used as a robust state estimator [1,34].
The article's non-linear optimal control method is a novel and significant result. Other approaches about the use of H-infinity control in non-linear dynamical systems were limited to the case of affine-in-the-input systems with drift-only dynamics and considered that the control inputs gain matrix is not dependent of the values of the system's state vector. Moreover, in these approaches, the linearisation was performed around points of the desirable trajectory whereas in this article's control method the linearisation points are related with the value of the state vector at each sampling instant, as well as with the last sampled value of the control inputs vector. The Riccati equation which has been proposed for computing the feedback gains of the controller is novel, so is the presented global stability proof through Lyapunov analysis. As it will be explained in the following sections, the presented non-linear optimal control method has improved performance when compared against other non-linear control schemes that one can consider for the dynamic model of electrohydraulic robotic manipulators.
The article's scientific contribution is outlined as follows: (i) the presented non-linear optimal control method has improved performance when compared against other non-linear control schemes that one can consider for the dynamic model of the multi-DOF electro-hydraulic robotic manipulator (such as Lie algebrabased control, differential flatness theory-based control, modelbased predictive control, non-linear model-based predictive control, sliding-mode control, backstepping control, (ii) it achieves fast and accurate tracking of all reference setpoints for the multi-DOF electro-hydraulic robotic manipulator under moderate variations of the control inputs, (iii) it minimises the consumption of energy by the multi-DOF electro-hydraulic robotic manipulator's actuators, thus improving the operational capacity and efficiency in tasks execution for this robotic system. The structure of this paper is organised as follows: in Section 2, the dynamic model of a multi-DOF (2-DOF) electro-hydraulic robotic manipulator is analysed. In Section 3, the differential flatness properties of the dynamic model of the robotic manipulator are proven. In Section 4, the state-space model of the electrohydraulic manipulator undergoes approximate linearisation with the use of first-order Taylor series expansion and through the computation of the related Jacobian matrices. Furthermore, a stabilising H-infinity (optimal) feedback controller is designed. In Section 5, the global stability properties of the proposed control method are proven through Lyapunov analysis. Besides, the Hinfinity Kalman filter is proposed as a robust observer which allows for the implementation of state estimation-based control. In Section 6, the performance of the non-linear optimal control method for electro-hydraulic manipulators is further confirmed through simulation experiments. Finally, in Section 7, concluding remarks are stated.

G(θ) is the gravitational forces vector
and τ(t) is the control inputs vector consisting of the torques that are generated by rotary electro-hydraulic actuators mounted on the robot's joints. The torques vector applied to the manipulator is τ = [τ 1 , τ 2 ] T . Moreover, if one considers elasticity in the second joint, the torques vector becomes τ = [τ 1 , τ 2 + k(θ 1 − θ 2 )] T . Torques Torques τ 1 and τ 2 are generated by rotary electro-hydraulic actuators and are given by where K P i , i = 1, 2 are force to torque conversion coefficients for the ith actuator, A P i , i = 1, 2 is the effective area of the cylinder's chamber of the ith actuator and P L i , i = 1, 2 is the differential load pressure of the cylinder of the ith actuator. The dynamics of the two hydraulic actuators are given by the following equations [8,23,24]: where x v i , i = 1, 2 is the position of the ith electrovalve. Next, the following state-variables are defined: With this notation of state-variables, the dynamic model of the system is written in the following form: Moreover, the state-space model of the electro-hydraulic manipulator can be written in the following matrix form: are given by

Differential flatness properties of the hydraulic robotic manipulator
It can be proven that the dynamic model of the electro-hydraulic robotic manipulator is differentially flat. The differential flatness property signifies that (i) all state variables of the system and its control inputs can be written as differential functions of a subset of its state variables which are the system's flat outputs, (ii) the flat outputs are not connected through a relation in the form of a homogeneous differential equation, thus being differentially independent [36]. The differential flatness property allows defining setpoints for the control loop of the robotic manipulator. Actually, one can primarily define setpoints for the state variables which stand for the flat outputs of the robotic system, and though they can compute the setpoints' functions for the rest of the state variables of the robot.
The flat outputs of the electro-hydraulic robotic manipulator are taken to be y 1 = x ! and y 2 = x 3 , that is the two turn angles of the robot's joints. From the first row of the state-space model, one has From the third row of the state-space model one has Thus, both state variables x 2 and x 4 are differential functions of the system's flat outputs. Next, it is confirmed that the elements of the inertia matrix M(θ), of the Coriolis matrix C(θ, θ˙) and of the gravitational matrix G(θ), as well as the determinant of the inertia matrix detM are written as differential functions of the flat outputs of the system. Additionally, from the second and fourth rows of the robot's state-space model, one has a system of equations with respect to state variables x 5 and x 7 . Thus one obtains or equivalently Consequently, it can be inferred that state variables x 5 and x 7 are also written as differential functions of the system's flat outputs. Moreover, from the fifth and seventh rows of the state-space model of the robotic manipulator one solves with respect to state variables x 6 and x 8 . Actually, from the fifth tow of the state-space model one has Next, by defining as h 1 (x, x˙) the fraction which appears in the righthand part of (28) and which does not include x 6 or x 8 , one obtains which signifies that state variable x 6 is a differential function of the system's flat outputs. Moreover, from the seventh row of the statespace model one obtains Next, by defining as h 2 (x, x˙) the fraction which appears in the righthand part of (28) and which does not include x 6 or x 8 , one obtains which signifies that state variable x 8 is a differential function of the system's flat outputs. Finally, from the sixth and eighth rows of the state-space model of the robotic manipulator, one can solve with respect to the control inputs u 1 and u 2 . This gives These equations come to complete the proof of the differential flatness property of the state-space model of the multi-DOF (2-DOF) electro-hydraulic robotic manipulator. By defining setpoints x 1 d and x 3 d for the state variables x 1 and x 3 which are also the flat outputs of the robotic model, one can compute next the functions of the setpoints of the rest of the state variables of the system, with the use of (24), (25), (27), (29) and (31).
Fourth row of the Jacobian matrix A = ∇ x [ f (x)] (x * , u * ) : By defining again N 1 = K P 1 A P 1 x 5 , N 2 = K P 2 A P 2 x 7 and the numerator function About the differentiation with respect to x i of the numerator where the above noted partial derivatives were computed before.

Stabilising feedback control
After linearisation around its current operating point, the dynamic model of the electro-hydraulic robotic manipulator is written as [1]  Parameter d 1 stands for the linearisation error in the electrohydraulic robotic manipulator's dynamic model appearing above in (42). The reference setpoints for the state vector of the electrohydraulic robotic manipulator are denoted by Tracking of this trajectory is achieved after applying the control input u * . At every time instant the control input u * is assumed to differ from the control input u appearing in (42) by an amount equal to Δu, that is u * = u + Δu The dynamics of the controlled system described in (42) can be also written as and by denoting d 3 = − Bu * + d 1 as an aggregate disturbance term one obtains By subtracting (43) from (45) one has By denoting the tracking error as e = x − x d and the aggregate disturbance term as d = d 3 − d 2 , the tracking error dynamics becomes For the approximately linearised model of the system a stabilising feedback controller is developed. The controller has the form with K = (1/r)B T P, where P is a positive definite symmetric matrix which is obtained from the solution of the Riccati equation [

1]
where Q is a positive semi-definite symmetric matrix. The diagram of the considered control loop is depicted in Fig. 2. The solution of the H-infinity feedback control problem for the multi-DOF electro-hydraulic robotic manipulator and the computation of the worst case disturbance that this controller can sustain, comes from superposition of Bellman's optimality principle when considering that the robot is affected by two separate inputs (i) the control input u (ii) the cumulative disturbance input d(t). Solving the optimal control problem for u that is for the minimum variation (optimal) control input that achieves elimination of the state vector's tracking error gives u = − (1/r)B T Pe. Equivalently, solving the optimal control problem for d, that is for the worst-case disturbance that the control loop can sustain gives d = (1/ ρ 2 )L T Pe.
A comparison of the proposed non-linear optimal (H-infinity) control method against other linear and non-linear control schemes for the electro-hydraulic robotic manipulator's dynamics, shows the following: (i) Unlike global linearisation-based control approaches, such as Lie algebra-based control and differential flatness theory-based control, the optimal control approach does not rely on complicated transformations (diffeomorphisms) of the system's state variables. Besides, the computed control inputs are applied directly on the initial nonlinear model of the electro-hydraulic robotic manipulator's dynamics and not on its linearised equivalent. The inverse transformations which are met in global linearisation-based control are avoided and consequently one does not come against the related singularity problems.
(ii) Unlike model predictive control (MPC) and non-linear MPC (NMPC), the proposed control method is of proven global stability. It is known that MPC is a linear control approach that if applied to the non-linear dynamics of the electro-hydraulic robotic manipulator the stability of the control loop will be lost. Besides, in NMPC the convergence of its iterative search for an optimum depends on initialisation and parameter values selection and consequently the global stability of this control method cannot be always assured. (iii) Unlike sliding mode control and backstepping control, the proposed optimal control method does not require the state-space description of the system to be found in a specific form. About sliding-mode control, it is known that when the controlled system is not found in the input-output linearised form the definition of the sliding surface can be an intuitive procedure. About backstepping control, it is known that it cannot be directly applied to a dynamical system if the related state-space model is not found in the triangular (backstepping integral) form. (iv) Unlike PID control, the proposed non-linear optimal control method is of proven global stability, the selection of the controller's parameters does not rely on a heuristic tuning procedure, and the stability of the control loop is assured in the case of changes of operating points.
(v) Unlike multiple local models-based control the non-linear optimal control method uses only one linearisation point and needs the solution of only one Riccati equation so as to compute the stabilising feedback gains of the controller. Consequently, in terms of computation load the proposed control method for the electrohydraulic robotic manipulator's dynamics is much more efficient.

Proof of global stability properties
Through Lyapunov stability analysis it will be shown that the proposed non-linear control scheme assures H ∞ tracking performance for the electro-hydraulic robotic manipulator's dynamics, and that under moderate conditions asymptotic convergence to the reference setpoints (global stability) is achieved [1]. As shown before, the tracking error dynamics for the electrohydraulic robotic manipulator's dynamics is written in the form [1] e˙= Ae + Bu + Ld (50) where in the electro-hydraulic robotic manipulator's case L = I ∈ R 8 × 8 with I being the identity matrix. Variable d denotes model uncertainties and external disturbances of the robotic manipulator's dynamic model. The following Lyapunov equation is considered: where e = x − x d is the tracking error. By differentiating with respect to time one obtains The previous equation is rewritten as (55) Assumption: For given positive definite matrix Q and coefficients r and ρ there exists a positive definite matrix P, which is the solution of the following matrix equation Moreover, the following feedback control law is applied to the system: By substituting (56) and (57) one obtains which after intermediate operations gives or, equivalently The following inequality holds: Expanding the left part of the above inequality one gets The following substitutions are carried out: a = d and b = e T PL and the previous relation becomes Equation (64) is substituted in (61) and the inequality is enforced, thus giving Equation (65) shows that the H ∞ tracking performance criterion is satisfied. The integration of V˙ from 0 to T gives Moreover, if there exists a positive constant M d > 0 such that then one gets Thus, the integral ∫ 0 ∞ ∥ e ∥ Q 2 dt is bounded. Moreover, V(T) is bounded and from the definition of the Lyapunov function V in (51) it becomes clear that e(t) will be also bounded since e(t) ∈ Ω e = {e e T Pe ≤ 2V(0) + ρ 2 M d }.
According to the above and with the use of Barbalat's lemma one obtains lim t → ∞ e(t) = 0.
The outline of the global stability proof is that at each iteration of the control algorithm the state vector of the electro-hydraulic robotic manipulator's dynamics converges towards the temporary equilibrium and the temporary equilibrium in turn converges towards the reference trajectory. Thus, the control scheme exhibits global asymptotic stability properties and not local stability [1]. Assume the ith iteration of the control algorithm and the ith time interval about which a positive definite symmetric matrix P is obtained from the solution of the Riccati equation appearing in (56). By following the stages of the stability proof one arrives at (65) which shows that the H-infinity tracking performance criterion holds. By selecting the attenuation coefficient ρ to be sufficiently small and in particular to satisfy ρ 2 < ∥ e ∥ Q 2 / ∥ d ∥ 2 one has that the first derivative of the Lyapunov function is upper bounded by 0. Therefore for the ith time interval it is proven that the Lyapunov function defined in (51) is a decreasing one. This signifies that between the beginning and the end of the ith time interval there will be a drop of the value of the Lyapunov function and since matrix P is a positive definite one, the only way for this to happen is the Euclidean norm of the state vector error e to be decreasing. This means that comparing to the beginning of each time interval, the distance of the state vector error from 0 at the end of the time interval has diminished. Consequently as the iterations of the control algorithm advance the tracking error will approach zero, and this is a global asymptotic stability condition. □

State estimation with robust Kalman filtering
The control loop has to be implemented with the use of information provided by a small number of sensors and by processing only a small number of state variables. To reconstruct the missing information about the state vector of the electro-hydraulic robotic manipulator's dynamics it is proposed to use a filtering scheme and based on it to apply state estimation-based control [1,36]. The recursion of the H ∞ Kalman filter, for the model of the electrohydraulic robotic manipulator's dynamics, can be formulated in terms of a measurement update and a time update part Measurement update Time update where it is assumed that parameter θ is sufficiently small to assure that the covariance matrix P − (k) −1 − θW(k) + C T (k)R(k) −1 C(k) will be positive definite. When θ = 0 the H ∞ Kalman filter becomes equivalent to the standard Kalman filter. One can measure only a part of the state vector of the electro-hydraulic robotic manipulator', such as state variables x 1 , x 3 , x 5 and x 7 , and can estimate through filtering the rest of the state vector elements. Moreover, the proposed Kalman filtering method can be used for sensor fusion purposes.

Simulation tests
The performance of the proposed nonlinear optimal control scheme for the multi-DOF (2-DOF) electro-hydraulic manipulator has been tested and confirmed through simulation experiments.  The transient performance of the proposed H-infinity control scheme is determined by the selection of the parameters r, ρ and Q which appear in the previously noted Riccati equation of (56). Actually, relatively small values of r result into elimination of the tracking error, while relatively large values of ρ result into fast convergence to the reference setpoints. Besides, the smallest value of ρ for which one can obtain a valid solution of the Riccati equation of (56), that is a positive definite symmetric matrix P, is the one that provides the control loop with maximum robustness. The proposed control method exhibits specific advantages, as for instance the avoidance of complicated state variables transformations, the existence of a global stability proof and computational efficiency because of needing only the solution of one algebraic Riccati equation.

Conclusions
Electro-hydraulic robots can find wide use in industry, in mining tasks and in construction works, and in general in applications where heavy loads have to be manipulated and high forces and torques have to be generated. Precision in the manipulations performed by these robots, as well as minimisation of energy consumption are the main objectives of the related control schemes. In this article, a novel nonlinear optimal (H-infinity) control method has been proposed for electro-hydraulic robotic manipulators. The method uses approximate linearisation around a temporary operating point which was re-computed at each timestep of the control algorithm. The linearisation process relies on first-order Taylor series expansion and on the computation of the related Jacobian matrices.
For the linearised state-space model of the robotic manipulator, a stabilising H-infinity (optimal) feedback controller was designed. The proposed H-infinity controller represents a min-max differential game taking place between the control inputs and the model uncertainty or external perturbation inputs. To compute the feedback gains of this controller, an algebraic Riccati equation was repetitively being solved at each time-step of the control algorithm. The global asymptotic stability properties of the control scheme have been proven through Lyapunov analysis. By proving the differential flatness properties of the robot's model the definition of setpoints for the optimal control loop was easily accomplished. Finally, to implement state estimation-based control for the electrohydraulic manipulator, the H-infinity Kalman filter has been used as a robust state observer.
The proposed non-linear optimal control method is generic and applicable to a wide class of robotic manipulators and robotic vehicles and particularly to those functioning in underactuation conditions. Other control methods for such robotic systems are suboptimal and select feedback control gains in an empirical and ad-hoc manner. Such methods can be Lie algebra-based control, differential flatness theory-based control, model-based predictive control, non-linear model-based predictive control, sliding-mode control, backstepping control, PID control and multiple linear models-based feedback control. Because the proposed control scheme retains the advantages of linear optimal control, it achieves rapid convergence of the robotic systems' state variables to the reference setpoints, and is characterised by small rise time, small settling time and hardly no overshoot. Such convergence properties are difficult to be achieved by the ad-hoc controller gains' selection that is performed by the rest of the aforementioned control methods. Besides, the proposed control scheme minimises the variations of the control inputs, which signifies that precise following of the reference paths is achieved under minimal energy consumption by the actuators of the robotic systems. This is something that the rest of the control methods cannot attain.