Distributed accelerated descent algorithm for energy resource coordination in multi-agent integrated energy systems

Composed of multiple integrated energy systems (IESs) belonging to different stakeholders, multi-agent IESs (MA-IESs) are widely concerned because of data privacy protect. As the basis of planning design and reliability evaluation for MA-IESs, distributed energy resource coordination (DERC) problem is studied in this paper. First, a DERC model for MA-IESs is established, which considers energy conversion process speciﬁcally. Meanwhile, a novel distributed accelerated descent (DAD) algorithm is proposed to realize fully distributed solving. Different from most of the existing researches that investigate the DERC with box constraints, the presented algorithm is able to solve the DERC with general convex constraints. Moreover, the backward operators in the method improve the convergence rate to the best of distributed ﬁrst-order optimization algorithm with ﬁxed step size, O (1 ∕ T ). Furthermore, the presented approach is initialization robustness when the load ﬂuctuations suddenly happened in MA-IESs. The convergence property, computing, and communication complexity are strictly proved. Finally, the effectiveness of DERC model and DAD algorithm are demonstrated by some modiﬁed case studies.


INTRODUCTION
In the recent years, the fossil energy crisis and environmental pollution are becoming more and more serious. Therefore, the efficiency of energy has drawn much research attention. In order to achieve the purpose of improving energy efficiency, integrated energy system (IES) [1] envisions a future energy system with much more interactions between multiple energies. As the basis of planning design and reliability evaluation for IESs, energy resource coordination (ERC) has been widely studied, which is one type of the economic dispatch problem (EDP). The most common method of ERC is to collect and calculate the data through the centralized independent system operator (ISO) [2], which is similar to the operation methods in traditional power system. However, with increasing system scale of network interlinks, more and more IESs belonging to different stakeholders participate in the ERC process. In this situation, the multi-agent IESs (MA-IESs) make the centralized approaches with one-toall modes meet a few drawbacks. in MA-IESs do not want to exchange core information because of data privacy and commercial interests. Meanwhile, with the application of artificial intelligence in MA-IESs, high volume data exchange between communication lines requires higher bandwidth, less single point failure, and faster calculation speed [3,4]. Therefore, establishing distributed ERC (DERC) models and algorithms for MA-IES have become crucial tasks, and related researches are in urgent need.
The most important characteristic for DERC problem is that each agent can only know its specific information, such as local objective function, local feasibility constraints, and local calculation data. They need to achieve global optimization through cooperation. According to these requirements, [5] studies a class of DERC problem with general linearly coupled constraints based on random updates. Ref. [6] established a DERC model and method of smart grid considering random wind power, in which the basic convergence condition of distributed algorithm has been verified. A distributed algorithm was proposed in [7], which improved the scalability of DERC algorithm and basically achieved the plug-and-play operation for smart grid.
The factor of time-varying communication networks in DERC problem was discussed in [8][9][10]. A distributed approach with quadratic convergence rate was proved in [8] on condition of long-term connectivity undirected graphs. Ref. [9] guaranteed a gradient push-sum-based algorithm considering time-varying delays, while [10] further considered the network uncertainties of DERC, where random communication networks were investigated. Although quite a few meaningful researches have been published, most of the papers above are applied in independent power systems or information systems. The researches on DERC problem for MA-IESs are still in early stage, where the modeling of energy conversion among different energy types has not been reported.
To address the DERC problem, a few distributed algorithms (see [11][12][13][14][15][16][17][18][19][20][21][22][23]) have been proposed. Based on alternating direction method of multiplier (ADMM), [11] presented an inexact update distributed method to deal with DERC problem, and [12] solves the problem with general convex objective function in islanded microgrids. But the transmission of Lagrange multipliers in these methods among large numbers of agents in MA-IESs cost higher communication. Refs. [13,14] improve the computation speed of distributed algorithms for DERC. Nevertheless, they need to be reinitialized, when the generation or load parameters change in MA-IESs [15]. The initialization procedure limits the scalability and intelligence of DIESs with integrated multiple energy and distributed resource energy. Initialization robust algorithms are analyzed in [16] and [17], the complex iteration forms of which make them difficult to apply in MA-IESs. A novel approach is presented to decrease computation time for a highly dynamic power distribution system in [18]. Ref. [19] presented an innovative dual-decomposition-based distributed algorithm to deal with micro-IIESs including multi-agent energy hubs. However, the relatively complicated distributed modeling process make them difficult to be satisfied in engineering application. Ref. [20] studied the distributed optimal power flow based on ADMM, which realized the fully distributed load flow calculation. Moreover, considering stochastic communication delay, [21] raised synchronous and asynchronous iterative algorithms to solve distributed optimal power flow problem. Meanwhile, a distributed algorithm for reactive power compensation is designed in [22], the computational efficiency of which is verified through standard distribution power system. Despite the improvements of the aforementioned distributed algorithms, some counterparts illustrated greatly challenged the application range. Paper [23] proposed an iterative ADMM method to control a multi-area integrated electricity-natural gas system, which is effective to deal with the non-convexity of integer variables. Even though these algorithms consider complex constraints in MA-IESs, they are not fully distributed methods because of the coordinated center requirement.
Therefore, a novel distributed accelerated descent (DAD) algorithm to solve refined DERC model for MA-IESs is proposed in this paper. The main contributions of this paper are as follows: (1) An explicit DERC model for MA-IESs is established, where the energy conversion and transmission processes are considered. Besides the traditional box constraints, the constraints of CHP and EB units in each agent of MA-IESs are taken into account. Furthermore, in order to facilitate the algorithm design, the DERC model is reformulated to a compact mathematical form with quadratic cost function and general convex constraints. (2) The DAD algorithm is proposed to achieve fully distributed solving, which improves convergence performance via backward operators. The argmin step in the presented approach makes it suitable to deal with complex constraints in DERC model. The robustness initialization and fixed step-size iteration process further promote the applied engineering meaning of the presented method in MA-IESs. (3) The strict mathematical proofs of O(1∕T ) (sublinear) convergence rate is provided, which is the best convergence rate for distributed first-order algorithms [24]. Meanwhile, the O(1∕ ) complexity of the proposed algorithm accounts for the communication and computational rounds.
The remainder of this paper is organized as follows: Section 2 introduces the structure and DREC model for MA-IESs. Section 3 describes the proposed DAD algorithm and analyzes the advantages of it. Strict convergence and complexity proof are also provided. Case studies and simulation results are presented in Section 4. Section 5 concludes this paper.

MA-IES STRUCTURE AND DERC MODELING
MA-IES is an urban energy supply system with different stakeholders, and each agent of IES contains multiple energy subsystems. This section describes basic structure and DERC model of MA-IES. In this model, detailed energy conversion devices in one agent and distributed energy coordination among different agents are taken into account.

Basic structure of MA-IES
In actual industrial production, IES always consist of multiple agents with their own managers. Each agent of MA-IES includes electricity, natural gas, and heat sub-systems. In order to protect data privacy of different stakeholders, different agents of MA-IES can only know their local parameters and exchange few information via communication topology. A typical structure of MA-IES is illustrated in Figure 1. As shown in Figure 1, the electricity, gas, and heat subsystems in one agent of MA-IES are interconnected with each other through energy conversion equipments, such as combined heat and power (CHP) units and electrical boiler (EB) units. It is worth to note that MA-IES is an urban distributed system, in which the line transmission limitation can hardly be reached. Meanwhile, energy supply-demand balance problem is mainly focused in this paper, which is very useful for planning and reliability evaluation of IES. Moreover, congestion problem is always simplified in the initial research process of the urban multi-agent IESs, such as communication control strategy in [25], optimal storage planning in [26] and reliability revaluation in [27]. Therefore, the structures and capacity constraints of line are ignored. The electricity sub-system consists of coal-fired generations (CGs), electricity loads and some renewable generations (RGs), such as photovoltaic cells and wind generations. The natural gas sub-system includes gas sources (GSs) and gas loads. The heat sub-system contains heat loads and is supplied by CHP and EB units. Considering the inefficient of heat energy transmission over long distance, it is assumed that only electricity and gas energy could be coordinated among different agents.

DERC model of MA-IES
Aiming at describing energy supply-demand balance problem specifically, a DERC model for MA-IES is introduced here.

Objective function
The objective function (1) of DERC model for MA-IES is to minimize the total energy cost, in the method of multiple agent cooperation.

Local constraints in each agent
For the electricity sub-system in one agent, the scheduled power of CGs are limited by the capacity constraints in (2).
where p c i,mmin and p c i,mmax are the min and max output of the mth CG in agent i.
For the natural gas sub-system in one agent, the GS outputs are limited by (3).
where g s i,mmin and p s i,mmax are the min and max output of the mth GS in agent i.
For the heat sub-system in one agent, the CHP and EB units are heat sources. The heat balance equation is shown in (4) and the stochastic heat loads are assumed to be average values in a dispatch interval (such as 1 day or 1 h).
where h CHP i,m represents the heat output of the mth CHP unit, and h EB i,m represents the heat output of the mth EB unit in agent i. D H i denotes the total heat load demand of agent i. CHP i , EB i are the CHP and EB units set.
Two typical energy conversion devices, CHP and EB units, are considered here. The feasible region of CHP units are established via polyhedrons, where both -back pressure and extraction condensing units are uniformly considered [28]. The constraints of them are given in (5)- (9). denotethe gas-to-heat and gas-to-power conversion coefficient of CHP unit. p EB i,m represents the power input of mth EB unit in each agent. EB is the conversion coefficient of EB unit. The other nomenclature represents the capacity limitation of CHP and EB units.

Energy coordination constraints
Electricity and gas energy could coordinate among different agents via transmission lines. The energy coordination constraints of MA-IES are presented in (10) and (11).
where D E i denotes the total electricity load demand and D G i denotes the total gas load demand in agent i. D R i represents the intermittent power output of all RGs in agent i. As the same with heat loads above, D R i,l , D G i,l , D E i,l are also considered as average constant values here.

Reformulation of DREC model
For the further distributed algorithm design, the DREC model is reformulated to a compact mathematical form here. First, we define a variable matrix x i and a load demand matrix d i as: where Let the Ω i be the set of all x i ∈ ℝ 2 for which (2)-(9) are satisfied. Then, the local constraints in each agent of MA-IES can be written as: Note that all constraints in Ω i are linear and the cost functions of MA-IES are quadratic, the DERC model is a quadratic optimal problem. In order to increase the generalization of the DERC model, we further assume the local constraints Ω i is general convex constraints. Based on this, the DERC model for MA-IES can be formulated as follows: where for each agent i, x i ∈ ℝ 2 is its local decision variable, f i ∶ ℝ 2 → ℝ is its local objective function, the non-empty closed and convex constraint set Ω i ∈ ℝ 2 is its local constraint set and d i ∈ ℝ 2 is the local energy demand, all of which can only be obtained by agent i only. Define (13) is not a separable problem since it has a common linear equality constraint. Therefore, the agents in MA-IES need to find its own decision variable x i within its own constraint x i ∈ Ω i through a common equality constraint ∑ n i=1 x i = ∑ n i=1 d i in order to minimize the sum of local cost functions, without knowing other agents' feasible sets.

DISTRIBUTED ALGORITHM FOR THE DERC
In this section, a DAD algorithm is designed for the DERC model, the whole procedure of which is fully distributed without any center central system. Meanwhile, the sublinear convergence and complexity results of it are given here. Moreover, the advantages of the proposed algorithm compared with existing distributed method are explained at the same time.

Preliminary and assumption
The communication topology between agents is described by an undirected G = (N , E ), where N = {1, 2, 3, … , n} is the agent set, E ∈ N × N is the edge sets of communicating agents. If agent i and agent j can communicate with each other, then there exists an edge from i to j which can be denoted by (i, j ) ∈ E. Also we assume that there exists a self-loop (i, i ) for all agents i ∈ N . Then, we can define the neighbourhood of agent i by  i : Then, the associated Laplacian matrix L ∈ ℝ n×n of G = (N , E ) can be given as follows: We further make the following assumption on graph and local objective functions: (a) There exists an optimal solution x * ∈ Ω n of problem (13).
Remark 2. Assumption 1 is a general assumption which guarantees the convergence and the network communication of distributed resource coordination designs [29].
From Theorems 3.25-3.27 in [30], the following lemma holds: Lemma 1. According to Lagrange multiplier method, roblem (13) can be written as Lagrange function form. With Assumption 1, x * ∈ Ω n is the optimal solution of Problem (13) if there exists a Lagrange multiplier According to Lemma 1, solving distributed problem (13) is equivalent to the follow saddle point problem (16): where y ∈ ℝ 2 is a common Lagrange multiplier. In a completely distributed setting, problem (16) is equivalent to the following problem where y i ∈ ℝ 2 is a Lagrange multiplier for agent i. Define Solving Problem (13) z = (z 1 , … , z n ) ∈ ℝ 2n are Lagrange multipliers associated with Ly = 0. The constraints Ly = 0 is a compact way of writing constraints y i = y j for all agents i and j .

Lemma 2.
With Assumption 1, Problem (18) has optimal solution xį f x * is the optimal solution of Problem (13).
Proof. Similarly according to Theorems in [30], x˛is the optimal solution of (18) if there exist lagrange multipliers (y˛, z˛) such that ▿F (x˛) + y˛= 0 2n , ⟨ L, Y˛⟩ = 0 2n , , thus the conclusion holds. □ Lemmas 1 and 2 guarantee that the solution of problem (18) (the distributed form) is the same as that of problem (13) (centralized one). Meanwhile, with Assumptions 1, Slater's constraint condition also holds for problem (18). Thus, the following conclusion, which is the foundation of distributed algorithm design, follows: Remark 3. There exists an optimal solution x * ∈ Ω n of problem (13) and there exists (y * , z * ) such that (x * , y * , z * ) is a saddle point of problem (18).

Distributed accelerated descent algorithm
In order to increase the convexity, the augmented Lagrange function of problem (18) is given as (19), which is the basis of DAD algorithm design.
where , , are augmented multipliers. According to Lemmas 1 and 2, the first-order condition (KKT condition) also states that 0 mn = ▿ x * L c (x, y, z) = ▿F (x * ) + y * , 0 mn = ▿ z * L c (x, y, z) = ⟨ L, y * ⟩ , Equations (20) guarantees that we can construct algorithm through min x,y,z = L c (x, y, z). The DAD algorithm updates its multipliers as follows: x k = arg min x L c (x, y k , z k ).
Equations (21) and (22) are the backward operators, which is the core of accelerated descent.

2: Update
x k i = arg min 3: Check the end condition 1 If the condition is satisfied, then the algorithm is terminated. Otherwise, k ∶= k + 1 and go to step 2.
According to the structure of the Laplacian matrix L and its deduced matrix L, each agent i's local update rule can be written separately in Algorithm 1.
In Algorithm 1, each agent i has two local sequences, the primal ones {x k i } and the dual ones {y k i , z k i }. {x k i } is the decision variable of agent i, {y k i } is a subvector of all dual variables, while {z k i } is the consistent variable of dual variables. Therefore, the following conclusion follows: Remark 4. Algorithm 1 is a fully distributed method for DERC model of MA-IES. Different agents only need to exchange few non-core data, thus the cost function, load demand, and operation state of each stakeholder are protected.
By selecting suitable step sizes , , , the sublinear O(1∕T ) convergence rate of DAD algorithm with fixed step sizes reflected by i is given in Theorem 1. Theorem 1. Let x * be an optimal point of (13), suppose that = 2, = 2‖L‖ 2 , = 2, for all iteration step k = 1, … , T . For any T ≥ 1, we have 1 T Proof. Its detailed proof is derive from the proof of Theorem 1 in [31], thus we only show the proof steps here.
□ Remark 5. According to Theorem 1, we can see that the DAD algorithm reaches the O(1∕T ) (sublinear) convergence for quadratic function and general convex constraints, which is the best convergence rate for distributed first-order algorithm. Moreover, from Theorem 1, the complexity of proposed method can also be obtained. The DAD algorithm gets anoptimal solution in O(1∕ ) iterations and requires O(1∕ ) interagent communications of problem (13).

Advantages of the DAD algorithm
There are many distributed algorithms for standard energy coordination model developed in the recent years. The representative approaches are distributed projected primal-dual (DPPD) algorithm in [16], ADMM standard algorithm in [12], and improved ADMM algorithm in [13]. Compared with DPPD algorithm, the proposed DAD method improved the convergence rate through the backward operators, (26) and (27). Meanwhile, due to the general convex local constraints in MA-IESs, the argmin step (30) in Algorithm 1 makes the presented algorithm easier to solve than gradient projection step in DPPD algorithm. Moreover, the fixed step-size selection of DAD algorithm is given in this paper, which is more convenient for industrial application. Compared with ADMM standard algorithm, the DAD algorithm decreases the communication and computational complexity without exchanging the global information ‖L‖. Compared with improved ADMM algorithm, the quadratic penality term ‖x i − r i +ẑ k i ‖ 2 is replaced by the Euclidean distance term ‖x i − x k−1 i ‖ 2 . Thus, the DAD approach could easily implement convergence through random initial value selection.
To sum it up, the DAD algorithm proposed in this paper has the same simple iterative scheme compared with the distributed algorithms widely used in the energy engineering field, such as the ADMM algorithm. Furthermore, the DAD algorithm reduces communication bandwidth requirements and removes the dependence of initialization.

CASE STUDIES
In this section, the proposed DERC model and distributed algorithm for MA-IESs are tested on three cases. The first case considers conventional linear constraints and is tested on a modified 4-agent IES, where the convergence situation of presented distributed approach is illustrated. Meanwhile the comparison between distributed and centralized coordination results is provided. Moreover, the operation of energy conversion device in IES is analyzed in the first case. The second case is utilized to demonstrate the effectiveness of the proposed distributed algorithm applied in DERC with general convex constraints. The initialization robustness of it is tested at the same time. In the third case, the scalability of proposed distributed algorithm is verified via a modified 50-agent IES. The convergence performance of presented approach are illustrated through comparison with previous algorithms. All the cases are performed on a personal computer with Intel Core i7-4790 CPU (3.60 GHz) and 8.00 GB RAM on the Windows 10 operating system using MATLAB R2017b and Gurobi 8.0.

Case 1: Test on the modified 4-agent IES
The single-line diagram of modified 4-agent IES is shown in Figure 2. Different agents exchange information through undirected communication topology, given in Figure 3. Each agent in the IES consists of electricity, gas and heat sub-systems, the RG output and load demand of which are listed in Table 1. Electrical energy is supplied via one CG and one RG. Gas energy is supplied via one GS. Heat energy is supplied via CHP or The specific parameters of GS and CG are given in Table 2. The parameters of CHP and EB units are shown in Table 3 it can be seen that the optimal solution p c i * and g s i * of DERC satisfied the capacity constraints, and the agents with lower energy price provide more electricity energy. Meanwhile, the operation cost of IES considering iteration steps is illustrated in Figure 4f. Hence, the presented DERC model and distributed algorithm are effective for the MA-IESs with linear feasible region.
The comparison of calculation precision between the proposed distributed algorithm and traditional centralized algorithm for MA-IES is shown as follows.
From the Table 4, it is easy to observe that distributed approach achieves nearly the same calculation precision with centralized approach. Even though the several iteration steps of distributed method increase the computing time, only less unimportant information need to be exchange through different agents in MA-IES. Compared with centralized method, private data of different stakeholders, such as cost function, customer characteristics and operation situation, are protected. Moreover, the upper control center collecting whole information need a high level connectivity communication topology, which is easily subject to a single-point failure. Therefore, the proposed distributed algorithm is further accorded with the practical industrial application for DERC in MA-IES. In order to further analyzethe presented DERC model and distributed algorithm, the optimal heat energy supply modes of four agents in the test system are shown in Figure 5.
Note that the heat energy in agent 2 is all supplied via EB unit and the heat energy in agent 3 is all supplied via CHP unit. The EB unit in agent 1 provides more heat energy than CHP unit, because it has higher energy conversion efficiency. Analogously, The CHP unit in agent 4 provides more heat energy. Meanwhile, the energy output of unit is limited by rated capacity effectively. Therefore, the proposed DERC model considering energy conversion equipments is more specific than traditional DERC model, and the proposed distributed algorithm can realize optimal solution.  three stages. The solutions of optimal variables may get negative values because of test constraints, which will not occur in the industrial applications. Hence, on the basis of the original linear local constraints, the distributed algorithm proposed in this paper is further applicable for DERC model with convex simplex constraints.

Case 3: Test on the modified 50-agent IES
In order to verify the scalability, the proposed distributed algorithm is tested on a modified MA-IES with 50 agents in this case. The cost functions of CG and GS have the same form as that in case 1. It is assumed that there are one CG and one GS in each agent. The parameters of CG in agent i belong to the range a i ∈ [0.001, 0.010], b i ∈ [1, 10], p c i min ∈ [0, 5], p c i max ∈ [20,25]. The parameters of GS in agent i belong to the range c i ∈ [20,30], g s i min = 0, g s i max ∈ [15,20]. The total electricity and gas demand of the modified MA-IES are D E = 200 MW, D G = 300 MW, respectively. As the same with case 2, we also assume ∑ p c i = D E and ∑ g s i = D G . The communication topology is given in Figure 7.
By implementing the proposed algorithm, the simulation results are illustrated in Figure 8. Figure 8a,b shows the convergence of primal residual and energy balance gap. Meanwhile, the convergence performance (‖x k − x * ‖ F ∕‖x 0 − x * ‖ F ) of the presented DAD algorithm and the DPPD algorithm of [16] are compared in Figure 8c. It is shown that the iteration number of DAD is 15009, while the iteration number of DPPD is 35001. DAD is more efficient and more oscillating than DPPD because of the backward operators (26) and (27). However, the oscillation is within the margin of error, which do not give influence to its application. Thus, the proposed distributed method in this paper is effective for a large-scale MA-IES and has better convergence performance.

CONCLUSION
In this paper, a DERC model for MA-IES is established, in which energy conversion is explicitly considered comparing with traditional DREC model. In order to protect data privacy, a novel DAD algorithm based on strict sublinear convergence proof is proposed to achieve distributed optimization, the communication and computational complexity O(1∕ ) of which are given at the same time. The simulation results further illustrate that the proposed distributed algorithm is effective for the general simplex constraints and is robust to the initialization. Moreover, the DAD algorithm has better convergence performance because of backward operator, which is more applicable for a large-scale MA-IES. It would like to note that many challenges of DERC model and DAD algorithm still remain, such as the transmission line constraints and the 0-1 integer variables. These challenges can be further studied based on our research.