Accelerated hierarchical optimization method for emergency energy management of microgrids with energy storage systems

When failures occur in microgrids (MGs), the energy management for emergencies is required. To respond to emergencies in MGs rapidly, an accelerated hierarchical optimization method has been proposed, where the outputs of energy storage systems (ESSs) are controlled to provide urgent supports, before the MG reconfiguration starts. However, it is time‐consuming to find the optimal schemes for MG reconfiguration. To reduce the time of reconfiguration, an optimization method based on deep neural networks (DNNs) for MG reconfiguration is presented, which consists of three levels. First, the combinational optimization for load shedding amounts is solved to determine the loads that have to be cut off. Second, a DNN is used to find the parameters of reconfiguration instead of the time‐consuming calculation of power flow, which is a good way to reduce time of reconfiguration. Third, according to these parameters, the optimal scheme for reconfiguration is selected by a comprehensive evaluation method, where the Delphi method (DM) is employed to adjust the weights of preferences in a comprehensive evaluation function, so it offers the diversity of decisions for the MG reconfiguration. Finally, to test our method, a modified IEEE 33‐bus system is built in MATLAB for simulations. Compared to traditional methods, our method can obtain the same reconfiguration scheme under different on/off states of load switches, but the time of reconfiguration is only one‐sixty‐seventh of that of other methods. Furthermore, in terms of our comprehensive evaluation method, reconfiguration schemes can be selected under different preferences.


| INTRODUCTION
In recent years, renewable power generation has attracted more attention, but the intermittence of renewable energy causes negative effects on the power quality of main grids. 1 Therefore, the concept of a microgrid (MG) is proposed, where an MG is a small power generation and distribution system that is integrated by distributed generators (DGs), loads, and energy storage systems (ESSs). 2,3 Generally speaking, MGs work in two modes: grid-connected and islanded modes. In the grid-connected mode, the MG can exchange power with the main grid to ensure the supply-demand balance. However, when the main grid fails, the MG can switch to islanded mode. At this time, if the supplydemand balance is broken, network reconfiguration will be needed to maintain and achieve the stability and security of the MG. 4,5 Generally, the network reconfiguration means the topology of a power system is changed by adjusting the on/off states of load switches under some constraints in order to reach certain optima of objective functions. 6 The difficulty lies in how to reduce time of reconfiguration and find the optimal reconfiguration. 7 Currently, three types of methods for reconfiguration have been studied, such as traditional optimization methods, [8][9][10][11][12][13][14][15][16] heuristic methods, 17,18 and artificial intelligence based methods. [19][20][21][22][23] For example, the reconfiguration and reactive power optimization were formulated in the form of mathematical programming, which transformed the problem into a form with a convex feasible region in order to solve it more conveniently. 8 Moreover, when the reconfiguration of distribution networks was needed, a ring network was formed first, and then, the on/off states of load switches were selected to transfer loads, which made the distribution of loads better and network losses lower. 9 To find the optimal schemes for the self-healing of electrical distribution systems, a method using the mixed-integer non-linear programming model was presented, where the branch flow and the bus injection were considered. 13 Further, a mixedinteger second-order cone programming method was adopted for service restoration of distribution networks with DGs, where the original non-convex power flow equations were relaxed into conic quadratic formats. 14,15 And a robust optimization restoration model based on a mixed-integer quadratic constraint programming was proposed to restore maximal outage loads in distribution networks. 16 Besides, heuristic methods and intelligent optimization algorithms also attract more attention. For instance, a meta-heuristic technique called the biogeographybased optimization (BBO) was used to find the optimal solution for network reconfiguration. 18 And a method based on the firefly algorithm (FA) was also employed to determine the optimal locations and sizes of DGs in distribution networks, when reconfiguration was needed. 19 In 21 a direction matrix was introduced to a particle swarm optimization (PSO) that was used to guide the particle speeds, and then, it was applied to find the optimal reconstruction of a distribution network. Similarly, an improved genetic algorithm was applied to find the optimal reconstruction, where the on/ off states of load switches were considered as chromosome genes. 22 Owing to the fast reasoning mechanism and object-oriented features, an expert system was developed to obtain more solutions for network reconfiguration. 24 And a framework for smart reconfiguration in smart distribution grids was presented to minimize the power losses, when a new optimization method called Big Bang-Big Crunch algorithm was applied. 25 To reduce the power losses in the process of network reconfiguration, a method combined the flower pollination algorithm was proposed. 26 In addition, hierarchical optimization methods were also used to complete reconfiguration, and good results were achieved. [27][28][29] For example, in order to reduce the computational complexity during network reconstruction, a two-stage decentralized method for network reconstruction was proposed, which divided the entire network into separated subnets, and performed optimal reconstruction of the subnets in the first stage. The second stage coordinated the configuration of each subnet to obtain the optimal configuration result of the entire network. 27 In 28 a hierarchical optimization method, which treated the processes without power flow calculation as the first stage, and those with the calculation as the second stage, was proposed to reduce the calculation and at the same time an accurate optimal solution was obtained. Further, the hierarchical optimization method was extended to dynamic reconfiguration for MGs, where vulnerability stratification-based strategies were considered to improve the reconfiguration speed. 29 In recent years, energy storage systems (ESSs) have emerged as an effective way to maintain system stability. 30 Therefore, when failures occur in MGs, ESSs can support the system immediately to keep MGs still run normally, which makes MG reconfiguration and large economic losses avoided. However, in some emergencies, MG reconfiguration is inevitable, even if ESSs are applied. It is worse that if the number of DGs and loads in an MG increases significantly, it will be much more difficult to find the optimal reconfiguration schemes due to combinatorial explosion.
In order to improve the responses to emergencies in MGs, an accelerated hierarchical optimization method is proposed, where ESSs inject or absorb power into/ from the MG first, when emergencies appear. If these emergencies last, MG reconfiguration will be unavoidable. So, to reduce the time of reconfiguration, that is, avoiding the time-consuming calculation of power flow in MGs, an optimization method based on deep neural networks (DNNs) for MG reconfiguration is presented, which consists of three levels. First, the combinational optimization for load shedding amounts is solved to determine the loads that have to be cut off. Second, a DNN-based method is used to find the parameters of reconfiguration such as network losses and voltages, instead of the time-consuming calculation of power flow. Third, in terms of these parameters, a comprehensive evaluation function is developed to find the optimal scheme for reconfiguration. Finally, a modified IEEE 33-bus system was established in MATLAB for simulations. Compared to existing methods, the accelerated hierarchical optimization method can obtain the same reconfiguration scheme, but the time of reconfiguration is only one-sixty-seventh of that of other methods. Furthermore, in terms of our comprehensive evaluation method, reconfiguration schemes can be selected under different preferences.
The rest of the paper is organized as follows. In Section 2, the accelerated hierarchical optimization method is proposed, where there are three levels for different optimization problems. Section 3 introduces the test platform, where the parameters of DGs and loads are listed. In Section 4, cases are designed to compare the reconfiguration accuracy and reconfiguration time with a typical method. Section 5 concludes the paper.

HIERARCHICAL OPTIMIZATION METHOD
In the hierarchical optimization method, there are three levels for different optimization problems. In the first level, the aim of optimization is to find the optimal outputs of ESSs and respond the emergency situation rapidly. If MG reconfiguration is inevitable, the last two levels will start. In the second level, the on/off states of load switches are obtained, when the load shedding amount is identified and the supply-demand balance in MGs is satisfied. In the third level, the trained DNN is used to obtain parameters of reconfiguration, when the on/off states of load switches are given to the DNN. And then a comprehensive evaluation method is proposed to find the optimal scheme of reconfiguration, where the Delphi method (DM) 31 is employed to adjust the weights of preferences in the comprehensive evaluation function.

| First level: Optimization model for ESSs
Assume an MG consists of micro-turbine (MTs), photovoltaics (PVs), wind turbines (WTs), and ESSs. Generally speaking, the stable operation of MGs is the basis of energy management. During normal operation, the main objective for energy management is to minimize the operation costs, and the ESSs work in charging or discharging mode according to the states of the MG operation. However, when emergency occurs in microgrids, the main objective is changed to maintain the stability of the system. If faults occur in the MG, its frequency and voltage will fluctuate. Even the stability will be destroyed because of the imbalance of the supply-demand. Fortunately, ESSs can respond rapidly to these emergency situations and inject power to maintain the system stability. Therefore, when emergencies occur, the optimization for outputs of ESSs is an important problem.
In this section, the model of ESSs is given first, and then, the optimization model for ESSs is developed. As is known, the energy E (t + 1) in an ESS at time t+1 is related to the energy E (t) at time t and the charging/discharging power P (t) in an interval Δt, which can be expressed as where c and d are the charging/discharging rates of the ESS. So, Equation (1) describes the charging/discharging behavior of an ESS. However, both the energy in an ESS and the charging/discharging power are limited, which satisfy the constraints as follows, where E min and E max are the minimal and maximal capacities of an ESS, and similarly, P min and P max are its minimal and maximal charging/discharging power.
Next, an optimization model is developed to find the optimal outputs of ESSs in emergency situations, where the operation costs of ESSs and the costs for load shedding are considered. Generally speaking, there are three types of loads in an MG according to their importance, that is, vital electrical loads (VeLs), essential electrical loads (EeLs), and normal electrical loads (NeLs), • Vital electrical loads (VeLs): Safety of personnel or serious damage will occur, if this type of loads is loss of power. • Essential electrical loads (EeLs): Loss of manufactured products will occur, if this type of loads is loss of power. • Normal electrical Loads (NeLs): No important effects on safety or production will occur, if this type of loads is loss of power.
Therefore, NeLs have the lowest costs compared to EeLs and VeLs, while the costs of EeLs and VeLs are very high, which results in the possibility of their being cut off is tiny. Consequently, the objective function can be written as where i and i are the cost coefficients for NeLs and outputs of ESSs, and S i (t) is the amount of a NeL i at time t . The mainly considered constraint is the constraint for the supply-demand balance, which means the generation power of all DGs is equal to the demand of all loads, while the other constraint is Equation (2). After solving this optimization model, the references for outputs of all ESSs are determined and the minimal amount of load shedding can be found.

| Second level: Combinational optimization for load shedding amounts
When MG reconfiguration starts, NeLs are considered to be cut off, while EeLs and VeLs are generally not cut off due to their high importance. In this section, the optimal combination for load shedding of NeLs is modeled and solved, when the load shedding amount S L is given. The minimal load shedding amount S Lmin has been determined in the previous section. If necessary, all NeLs can be cut off for stably running of the MG. So, the maximal load shedding amount is where m is the number of NeLs. Thus, the load shedding amount S L can be selected in an interval, S Lmin ≤ S L ≤ S Lmax , one of which can be expressed as where S L (j) denotes the selected load shedding amount, S Lmin and S Lmax denote the lower and upper limits of load shedding amounts, and n denotes the number of load shedding amounts.
Assume there are m NeLs, {S i |i = 1, …, m}. When the selected load shedding amount S L (j) is given, the optimization problem is to find the sum of a set of loads satisfies S L (j), which can be expressed as where a ij denotes the on or off state of a load switch i under the load shedding amount S L (j), which only takes a ij = 0 or 1. This optimization problem can be solved by the Zero-One Integer Programming (ZOIP) 32 and its results are a set of values of a ij , namely a set of on/off states of load switches { a 11 , …, a ij , …, a mj } . It is worth noting that it is very possible to find several sets of on/off states of load switches under the S L (j). Next, the obtained sets of on/off states of load switches are listed in a matrix D as rows. Repeating above mentioned steps till j = n, we can create the matrix D as follows, where all possible on/off states of load switches are found for all S L (j) , j = 1, . . . , n.

| Third level: finding optimal scheme for MG reconfiguration
In this section, the structure and the training data of the DNN are introduced first, and then, the trained DNN is used to obtain parameters of reconfiguration instead of calculating power flow. Finally, a comprehensive evaluation method is proposed to find the optimal scheme of MG reconfiguration.

| Parameters of reconfiguration from DNN
Deep neural networks (DNNs) 33 are one type of artificial neural networks (ANNs) with multiple hidden layers between the input and output layers. To reduce the time of reconfiguration, a DNN-based method is presented, where the DNN with two hidden layers is constructed as shown in Figure 1, and the number of hidden layers is determined by an empirical equation in 34 Before the DNN is used, it has to be trained first according to the data acquired from an MG.
To collect the data, an MG is established in MATLAB and the Matpower toolbox is used to calculate the power flow under a set of states of load switches, where the parameters of power flow include the network losses, node voltages, and slack bus power. The obtained data are randomly divided into two sets, the training set and the testing set, where the training set has 80% of the data, and the testing set has the remaining 20% of the data, and there is no data overlapping between the two sets.
A two-stage method is applied to train a DNN, that is, pre-trained and fine-tuned. In the pre-trained stage, the greedy unsupervised learning algorithm 35 is used in the network training process, whose target is obtaining the initial parameters of the DNN model, such as the weights and biases of neurons among all layers. However, these initial values of the DNN model should be fine-tuned to obtain the optimal values. So, the backpropagation algorithm 36 is used to optimize the model parameters, which improves the performance of the entire network. Finally, the DNN is validated by using the data in the testing set.
When the trained DNN is used, the on/off states of load switches are given to the DNN as input, while the slack bus power, network losses, and the maximal and minimal voltages are obtained as the outputs of the DNN, which can be expressed as where [ represents the outputs of the DNN, where P s (t) is the output power of the slack bus and P n (t) is the power of network losses, while U max and U min are the maximal and minimal voltages.

| Comprehensive evaluation of optimal schemes of reconfiguration
For a load shedding amount S L (j), if more than one set of states of load switches is found, these sets of states will be given to the trained DNN one by one, and different parameters of power flow will be obtained. So, in this section, a comprehensive evaluation method is developed to find the optimal set of states of load switches according to the outputs of DNN, that is, parameters of power flow. Moreover, some constraints have to be satisfied at the same time.
For the convenience of evaluation, the outputs of the DNN (the parameters of power flow) are normalized as follows: where P * n (t) denotes the reference value of the network loss, which can be set at the maximum of the measured network losses or a value greater than the maximum; ΔU denotes the voltage deviation, namely the difference between the highest and lowest voltages in the entire system; and U * denotes the reference voltage, while ΔP s (t) denotes the power deviation, namely the difference between the slack bus power and the reference power, and P * s (t) denotes the average value of the upper and lower limits of the output power of the slack bus.
Next, a comprehensive evaluation function is developed in order to find the optimal schemes of reconfiguration according to normalized parameters and decision preference. So, it has the form as below, where P n (t) ,Ũ, and P s (t) denote the normalized values of the network loss, voltage deviation, and slack bus power deviation, respectively, while k 1 , k 2 , and k 3 denote the decision preferences. Here, the preferences satisfy the following conditions, 0 < k 1 , k 2 , k 3 < 1 and k 1 + k 2 + k 3 = 1.
Furthermore, the preferences can be determined according to the Delphi method (DM). 31 The Delphi method refers to the evaluation experts to give their opinions on an issue by anonymous. After many rounds, their opinions are summarized as the result of preferences. Thus, the weights in the comprehensive evaluation function can be determined by the DM. When the optimization model Equation (9) is solved, it has to satisfy some constraints of the MG, where the main constraints are listed as follows.
1. Constraint for slack bus powers, which has form as where P i denotes the active power of the slack bus, and P imin and P imax denote the upper and the lower limits of the active power of the node i.

Constraint for node voltages, which takes form as
where U i denotes the voltage of node i, and U imin and U imax denote the upper and lower limits of the voltage of the node i.
After solving the optimization model, the scheme with the smallest value of the comprehensive evaluation function is selected as the optimal solution.

| SYSTEM ARCHITECTURE AND SETUPS
A modified IEEE 33-bus system is built as the test bed as shown in Figure 2, where there are a 1.5-MW MT at node 1, a 2-MW PV at node 3, a 1-MW WT at node 6, three ESSs (1.5 MW, 1 MW, and 2 MW) at nodes 12, 22, and 29, respectively, and loads at other nodes. Among them, the MT works as a slack bus of the system, PVs and WTs work in the constant power control mode, and ESSs work in the active and reactive power control mode. Moreover, the output range of the MT is 0-1.5 MW, the normal range of a node voltage is 0.9-1.1, and the grid voltage reference is 12.66 kV. Finally, the branch impedance and parameters of the system are listed in Table 1. The parameters in Equations (1) and (3) are listed in Table 2.

| SIMULATION RESULTS
To verify the proposed hierarchical optimization method for emergency energy management of MGs, a modified IEEE 33-bus system is built in MATLAB for simulations. Moreover, three cases are designed to evaluate the performance of the proposed method, and then, simulation results are analyzed, discussed, and compared.

| Case 1: Training and testing results of DNN
As mentioned in Section 2.3, a DNN with two hidden layers is constructed, where the inputs are a set of on/off states of load switches, namely a row of the matrix D, and the outputs are the parameters of reconfiguration, such as the slack bus power, network losses, and the maximal and minimal voltages. The training data are collected from an MG as shown in Figure 2, where there are m = 7 NeLs. Therefore, the possible combination of on/off states of load switches is 2 7 = 128, one of which can find a set of parameters of reconfiguration. Consequently, a dataset with 128 data can be obtained, in which 100 data are used as training data, while the remaining 28 data are used as testing data. For training the DNN, the sample batch is set at 20, the maximal number of training epochs is set at 5000, and the learning rate is set at 0.9 for better learning performance of the network training. Further, the sigmoid function is used as an activation function for the hidden layer, while the identity function is used as a linear activation function for the output layer.
In the training process, the training errors are defined as the differences between the true values and the output values of the DNN, which are shown in Figure 3. From Figure 3, it can be seen that the training errors are bounded in an interval of 1%, which satisfies the training requirements.
After the training is completed, the DNN is tested by using the testing set and the testing errors are defined as the differences between the true values and the outputs of the trained DNN. As shown in Figure 4, the testing errors are often less than 1%, which also satisfies the testing requirements.

| Case 2: Comparisons of reconfiguration accuracy and reconfiguration time
Assuming there is a failure in the MG. Immediately, ESSs at nodes 12, 22, and 29 inject power to the MG according to the optimal outputs from the optimization model, namely Equation (3). Moreover, the optimization model also gives the load shedding amount is S Lmin = 0.5 MW, while the maximal amount of NeLs is S Lmax = 0.7 MW in the MG. When a load shedding amount S L (j) is given, the combination of loads can be determined by solving Equation (5), which also means the on/off states of load switches are found, as shown in Table 3.
Next, the on/off states of load switches are given to the trained DNN as inputs, and then, the parameters of reconfiguration are obtained, which are listed in Table 4.
As shown in Table 4, when the load shedding amounts are from 0.5 MW to 0.6 MW, the required outputs of the slack bus are greater than its maximal outputs, so these load shedding amounts do not satisfy the constraint for the supply-demand balance. On the contrary, the schemes for reconfiguration are feasible, only when load shedding amount is 0.7 MW. However, there are five sets of states of load switches (five schemes), so they have to be evaluated in terms of Equation (9). When the preferences are set at [ k 1 k 2 k 3 ] = [0.8 0.1 0.1], the comprehensive evaluation results of three schemes are listed in Table 5.
As shown in the second column, the scheme with the minimal value of comprehensive evaluation results is the optimal solution, that is, the state of load switches, [0 0 1 0 0 1 0], is the optimal scheme for load shedding, which means loads at nodes 16 and 25 will be cut off. Finally, the reconfiguration time (RT) of our method (M1) is calculated, which is 0.107 s.
To compare our method with the traditional hierarchical optimization method (M2), 28 the results obtained by the traditional hierarchical optimization method are shown in the second row of Table 6.
Comparing the results obtained by our method and the other in Table 6, it can be see that the found optimal scheme is the same, but the reconfiguration time of the proposed method is only one-sixty-seventh of that of the traditional hierarchical optimization method. Therefore, the reconfiguration time is significantly improved by our method because a DNN-based method is applied.

| Case 3: Impacts of preferences on comprehensive evaluation method
Our method also can give the evaluation results in terms of different preferences, when there are many schemes for reconfiguration. To obtain more schemes for reconfiguration, we assume the load shedding amount is 1.7 MW. According to our method, there are seven schemes for reconfiguration and their parameters of reconfiguration are listed in Table 7. When the preferences is set at [ k 1 k 2 k 3 ] = [0.12 0.75 0.13], which means the voltage quality is giving the highest priority, the optimal solution with minimal value of evaluation results is shown in Table 8.
On the contrary, if the power deviation of the slack bus is giving the highest priority, that is, the preferences are [ k 1 k 2 k 3 ] = [0.12 0.11 0.77], then the optimal solution is shown in Table 9.
In summary, under different preferences, different schemes for reconfiguration can be obtained and selected, which offers the diversity of schemes for reconfiguration.

| CONCLUSION
In this paper, an accelerated hierarchical optimization method for emergency energy management is proposed, whose aim is to reduce the reconfiguration time. In an MG with ESSs, when emergency situations appear, ESSs are regulated according to an optimization model in order to support the system rapidly. After these operations, sometimes MG reconfiguration is still inevitable, so an optimization method with three levels is presented. In the first level, the on/off states of load switches are obtained and at the same time the supply-demand balance is satisfied in the MG. In the second level, the trained DNN is used to obtain parameters of power flow to reduce time of reconfiguration. In the third level, a comprehensive evaluation method is also proposed to find the optimal set of states of load switches, where the DM is employed to adjust the weights of preferences in the comprehensive evaluation function.
To verify the effectiveness of the proposed method, the modified IEEE 33-bus system is developed in MATLAB and used in simulations. After training and testing the DNN according to the data from the system, the training errors and the testing errors all are less than ± 1%, which indicate the requirements are satisfied. Further, the proposed method is compared with the traditional method, where the power flow is calculated for different on/off states of load switches. The comparison results show that the proposed method can obtain the same reconfiguration schemes as the traditional method, but its reconfiguration time is only one-sixty-seventh of that of the other method. Furthermore, in terms of our comprehensive evaluation method, the reconfiguration schemes can be obtained and selected under different preferences, which offers the diversity of schemes.